2024-03-16T20:31:23.431 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-16T20:31:23.431 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} --archive /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602163 --name leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi --owner scheduled_leonidus@teuthology --verbose -- /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602163/orig.config.yaml 2024-03-16T20:31:23.472 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T20:31:23.541 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602163 branch: wip-lusov-qdb-squid description: fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} email: null first_in_suite: false job_id: '7602163' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-lusov-qdb-squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-qdb-squid sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b owner: scheduled_leonidus@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - osd.4 - osd.5 - osd.6 - osd.7 seed: 6918 sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b sleep_before_teardown: 0 subset: 111/10000 suite: fs suite_branch: wip-lusov-qdb-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEllsH28MeQ1SW8dxOYFtPgzrJX/rVVTPJCd3FSmYFmusoy8payGfi9Hln4nFDY1GMRB7WRpogcPhOEfv/LrjLw= smithi195.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGktsF1WOWYs6IgkXWlub27PjDKbkq2fNseBYcLGWD6zOSTVzBaG1oqXQzKXi3cW65Xd617wzRsqSvsyiHVh6yc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_cephfs_shell teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-15_11:44:52 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-16T20:31:23.541 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa; will attempt to use it 2024-03-16T20:31:23.542 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa/tasks 2024-03-16T20:31:23.542 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-16T20:31:23.544 INFO:teuthology.task.internal:Checking packages... 2024-03-16T20:31:23.565 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b' 2024-03-16T20:31:23.565 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-16T20:31:23.565 INFO:teuthology.packaging:ref: None 2024-03-16T20:31:23.565 INFO:teuthology.packaging:tag: None 2024-03-16T20:31:23.565 INFO:teuthology.packaging:branch: wip-lusov-qdb-squid 2024-03-16T20:31:23.565 INFO:teuthology.packaging:sha1: aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-16T20:31:23.566 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-lusov-qdb-squid 2024-03-16T20:31:23.704 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1681-gaba3c1d2-1jammy 2024-03-16T20:31:23.705 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-16T20:31:23.717 INFO:teuthology.task.internal:no buildpackages task found 2024-03-16T20:31:23.717 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-16T20:31:23.731 INFO:teuthology.task.internal:Saving configuration 2024-03-16T20:31:23.746 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-16T20:31:23.755 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T20:31:23.780 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602163', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 20:25:52.594617', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIO4/5xNtvZVHcrpZYHgAa75P/9T5DE3Bb89zCiqGtX49'} 2024-03-16T20:31:23.804 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602163', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 20:25:52.596358', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFueH/NhY5eOErWJBJPoSu56NXQzG8Y0kWyVGpKxIq0i'} 2024-03-16T20:31:23.804 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-16T20:31:23.813 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-03-16T20:31:23.813 INFO:teuthology.task.internal:roles: ubuntu@smithi195.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-03-16T20:31:23.813 INFO:teuthology.run_tasks:Running task console_log... 2024-03-16T20:31:23.891 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7d0e983280>, signals=[15]) 2024-03-16T20:31:23.892 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-16T20:31:23.903 INFO:teuthology.task.internal:Opening connections... 2024-03-16T20:31:23.903 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-03-16T20:31:23.905 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T20:31:23.982 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi195.front.sepia.ceph.com 2024-03-16T20:31:23.982 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T20:31:24.058 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-16T20:31:24.070 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-03-16T20:31:24.075 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-03-16T20:31:24.076 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-03-16T20:31:24.121 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-16T20:31:24.122 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="Ubuntu" 2024-03-16T20:31:24.122 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="22.04" 2024-03-16T20:31:24.122 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-16T20:31:24.122 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_CODENAME=jammy 2024-03-16T20:31:24.122 INFO:teuthology.orchestra.run.smithi122.stdout:ID=ubuntu 2024-03-16T20:31:24.122 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE=debian 2024-03-16T20:31:24.122 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-16T20:31:24.122 INFO:teuthology.orchestra.run.smithi122.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-16T20:31:24.122 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-16T20:31:24.122 INFO:teuthology.orchestra.run.smithi122.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-16T20:31:24.122 INFO:teuthology.orchestra.run.smithi122.stdout:UBUNTU_CODENAME=jammy 2024-03-16T20:31:24.123 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-03-16T20:31:24.143 DEBUG:teuthology.orchestra.run.smithi195:> uname -m 2024-03-16T20:31:24.148 INFO:teuthology.orchestra.run.smithi195.stdout:x86_64 2024-03-16T20:31:24.148 DEBUG:teuthology.orchestra.run.smithi195:> cat /etc/os-release 2024-03-16T20:31:24.200 INFO:teuthology.orchestra.run.smithi195.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-16T20:31:24.200 INFO:teuthology.orchestra.run.smithi195.stdout:NAME="Ubuntu" 2024-03-16T20:31:24.201 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_ID="22.04" 2024-03-16T20:31:24.201 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-16T20:31:24.201 INFO:teuthology.orchestra.run.smithi195.stdout:VERSION_CODENAME=jammy 2024-03-16T20:31:24.201 INFO:teuthology.orchestra.run.smithi195.stdout:ID=ubuntu 2024-03-16T20:31:24.201 INFO:teuthology.orchestra.run.smithi195.stdout:ID_LIKE=debian 2024-03-16T20:31:24.201 INFO:teuthology.orchestra.run.smithi195.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-16T20:31:24.201 INFO:teuthology.orchestra.run.smithi195.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-16T20:31:24.201 INFO:teuthology.orchestra.run.smithi195.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-16T20:31:24.201 INFO:teuthology.orchestra.run.smithi195.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-16T20:31:24.201 INFO:teuthology.orchestra.run.smithi195.stdout:UBUNTU_CODENAME=jammy 2024-03-16T20:31:24.201 INFO:teuthology.lock.ops:Updating smithi195.front.sepia.ceph.com on lock server 2024-03-16T20:31:24.226 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-16T20:31:24.238 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-16T20:31:24.251 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-16T20:31:24.251 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-03-16T20:31:24.254 DEBUG:teuthology.orchestra.run.smithi195:> test '!' -e /home/ubuntu/cephtest 2024-03-16T20:31:24.258 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-16T20:31:24.268 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-16T20:31:24.268 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-03-16T20:31:24.303 DEBUG:teuthology.orchestra.run.smithi195:> test -z $(ls -A /var/lib/ceph) 2024-03-16T20:31:24.327 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-16T20:31:24.385 INFO:teuthology.run_tasks:Running task kernel... 2024-03-16T20:31:24.403 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-16T20:31:24.403 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-16T20:31:24.403 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T20:31:24.404 DEBUG:teuthology.orchestra.run.smithi195:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-16T20:31:24.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T20:31:24.408 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-03-16T20:31:24.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T20:31:24.409 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-03-16T20:31:24.454 INFO:teuthology.orchestra.run.smithi122.stdout:5.15.0-56-generic 2024-03-16T20:31:24.454 INFO:teuthology.task.kernel:Running kernel on smithi122: 5.15.0-56-generic 2024-03-16T20:31:24.454 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get clean 2024-03-16T20:31:24.457 INFO:teuthology.orchestra.run.smithi195.stdout:5.15.0-56-generic 2024-03-16T20:31:24.457 INFO:teuthology.task.kernel:Running kernel on smithi195: 5.15.0-56-generic 2024-03-16T20:31:24.457 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get clean 2024-03-16T20:31:24.604 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2024-03-16T20:31:24.607 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-03-16T20:31:24.746 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-16T20:31:24.752 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-16T20:31:24.771 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-16T20:31:24.773 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-16T20:31:24.849 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-16T20:31:24.855 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-16T20:31:24.859 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-16T20:31:24.860 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-16T20:31:24.974 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,462 kB] 2024-03-16T20:31:24.994 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [587 kB] 2024-03-16T20:31:25.074 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,462 kB] 2024-03-16T20:31:25.082 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [587 kB] 2024-03-16T20:31:25.091 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [285 kB] 2024-03-16T20:31:25.094 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-16T20:31:25.095 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-16T20:31:25.095 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,559 kB] 2024-03-16T20:31:25.118 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [285 kB] 2024-03-16T20:31:25.120 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [259 kB] 2024-03-16T20:31:25.124 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-16T20:31:25.124 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,057 kB] 2024-03-16T20:31:25.126 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-16T20:31:25.127 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-16T20:31:25.128 INFO:teuthology.orchestra.run.smithi122.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,559 kB] 2024-03-16T20:31:25.142 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [695 kB] 2024-03-16T20:31:25.153 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [239 kB] 2024-03-16T20:31:25.159 INFO:teuthology.orchestra.run.smithi122.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [259 kB] 2024-03-16T20:31:25.163 INFO:teuthology.orchestra.run.smithi122.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-16T20:31:25.163 INFO:teuthology.orchestra.run.smithi122.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [695 kB] 2024-03-16T20:31:25.170 INFO:teuthology.orchestra.run.smithi122.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,057 kB] 2024-03-16T20:31:25.172 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-16T20:31:25.172 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-16T20:31:25.173 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-16T20:31:25.173 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-16T20:31:25.174 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-16T20:31:25.188 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-03-16T20:31:25.189 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-03-16T20:31:25.189 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-03-16T20:31:25.190 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-16T20:31:25.190 INFO:teuthology.orchestra.run.smithi122.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [239 kB] 2024-03-16T20:31:25.192 INFO:teuthology.orchestra.run.smithi122.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-16T20:31:25.192 INFO:teuthology.orchestra.run.smithi122.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-16T20:31:25.192 INFO:teuthology.orchestra.run.smithi122.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-16T20:31:25.192 INFO:teuthology.orchestra.run.smithi122.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-16T20:31:25.206 INFO:teuthology.orchestra.run.smithi122.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-16T20:31:25.206 INFO:teuthology.orchestra.run.smithi122.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-03-16T20:31:25.206 INFO:teuthology.orchestra.run.smithi122.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-03-16T20:31:25.207 INFO:teuthology.orchestra.run.smithi122.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-03-16T20:31:25.207 INFO:teuthology.orchestra.run.smithi122.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-16T20:31:25.208 INFO:teuthology.orchestra.run.smithi195.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-03-16T20:31:25.209 INFO:teuthology.orchestra.run.smithi195.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-03-16T20:31:25.209 INFO:teuthology.orchestra.run.smithi195.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.6 kB] 2024-03-16T20:31:25.209 INFO:teuthology.orchestra.run.smithi195.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-16T20:31:25.225 INFO:teuthology.orchestra.run.smithi122.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-03-16T20:31:25.225 INFO:teuthology.orchestra.run.smithi122.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-03-16T20:31:25.226 INFO:teuthology.orchestra.run.smithi122.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.6 kB] 2024-03-16T20:31:25.226 INFO:teuthology.orchestra.run.smithi122.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-16T20:31:25.279 INFO:teuthology.orchestra.run.smithi195.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,246 kB] 2024-03-16T20:31:25.299 INFO:teuthology.orchestra.run.smithi122.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,246 kB] 2024-03-16T20:31:25.678 INFO:teuthology.orchestra.run.smithi195.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [421 kB] 2024-03-16T20:31:25.696 INFO:teuthology.orchestra.run.smithi195.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [225 kB] 2024-03-16T20:31:25.698 INFO:teuthology.orchestra.run.smithi122.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [421 kB] 2024-03-16T20:31:25.706 INFO:teuthology.orchestra.run.smithi195.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-16T20:31:25.706 INFO:teuthology.orchestra.run.smithi195.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,531 kB] 2024-03-16T20:31:25.716 INFO:teuthology.orchestra.run.smithi122.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [225 kB] 2024-03-16T20:31:25.726 INFO:teuthology.orchestra.run.smithi122.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-16T20:31:25.727 INFO:teuthology.orchestra.run.smithi122.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,531 kB] 2024-03-16T20:31:25.797 INFO:teuthology.orchestra.run.smithi195.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-16T20:31:25.798 INFO:teuthology.orchestra.run.smithi195.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [255 kB] 2024-03-16T20:31:25.809 INFO:teuthology.orchestra.run.smithi195.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-16T20:31:25.810 INFO:teuthology.orchestra.run.smithi195.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [598 kB] 2024-03-16T20:31:25.818 INFO:teuthology.orchestra.run.smithi122.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-16T20:31:25.819 INFO:teuthology.orchestra.run.smithi122.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [255 kB] 2024-03-16T20:31:25.830 INFO:teuthology.orchestra.run.smithi122.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-16T20:31:25.831 INFO:teuthology.orchestra.run.smithi122.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [598 kB] 2024-03-16T20:31:25.832 INFO:teuthology.orchestra.run.smithi195.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [850 kB] 2024-03-16T20:31:25.853 INFO:teuthology.orchestra.run.smithi122.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [850 kB] 2024-03-16T20:31:26.018 INFO:teuthology.orchestra.run.smithi195.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-03-16T20:31:26.039 INFO:teuthology.orchestra.run.smithi122.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-03-16T20:31:26.422 INFO:teuthology.orchestra.run.smithi195.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-16T20:31:26.428 INFO:teuthology.orchestra.run.smithi195.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-16T20:31:26.428 INFO:teuthology.orchestra.run.smithi195.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-16T20:31:26.442 INFO:teuthology.orchestra.run.smithi195.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-16T20:31:26.444 INFO:teuthology.orchestra.run.smithi195.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-16T20:31:26.446 INFO:teuthology.orchestra.run.smithi122.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-16T20:31:26.451 INFO:teuthology.orchestra.run.smithi122.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-16T20:31:26.466 INFO:teuthology.orchestra.run.smithi122.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-16T20:31:26.466 INFO:teuthology.orchestra.run.smithi122.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-16T20:31:26.469 INFO:teuthology.orchestra.run.smithi122.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-16T20:31:30.777 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 12.2 MB in 2s (6,810 kB/s) 2024-03-16T20:31:30.936 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 12.2 MB in 2s (6,900 kB/s) 2024-03-16T20:31:32.347 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-03-16T20:31:32.365 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-16T20:31:32.378 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-03-16T20:31:32.396 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-16T20:31:32.420 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-03-16T20:31:32.448 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-03-16T20:31:32.606 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-03-16T20:31:32.606 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-03-16T20:31:32.622 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-03-16T20:31:32.622 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-03-16T20:31:32.755 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-03-16T20:31:32.755 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-16T20:31:32.755 INFO:teuthology.orchestra.run.smithi122.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-16T20:31:32.756 INFO:teuthology.orchestra.run.smithi122.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-16T20:31:32.756 INFO:teuthology.orchestra.run.smithi122.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-16T20:31:32.756 INFO:teuthology.orchestra.run.smithi122.stdout: python2.7-minimal 2024-03-16T20:31:32.756 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-16T20:31:32.757 INFO:teuthology.orchestra.run.smithi122.stdout:The following additional packages will be installed: 2024-03-16T20:31:32.757 INFO:teuthology.orchestra.run.smithi122.stdout: linux-generic linux-headers-5.15.0-100 linux-headers-5.15.0-100-generic 2024-03-16T20:31:32.757 INFO:teuthology.orchestra.run.smithi122.stdout: linux-headers-generic linux-image-5.15.0-100-generic 2024-03-16T20:31:32.757 INFO:teuthology.orchestra.run.smithi122.stdout: linux-modules-5.15.0-100-generic linux-modules-extra-5.15.0-100-generic 2024-03-16T20:31:32.758 INFO:teuthology.orchestra.run.smithi122.stdout:Suggested packages: 2024-03-16T20:31:32.758 INFO:teuthology.orchestra.run.smithi122.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-16T20:31:32.758 INFO:teuthology.orchestra.run.smithi122.stdout:Recommended packages: 2024-03-16T20:31:32.759 INFO:teuthology.orchestra.run.smithi122.stdout: thermald 2024-03-16T20:31:32.774 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2024-03-16T20:31:32.774 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-16T20:31:32.774 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-16T20:31:32.774 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-16T20:31:32.774 INFO:teuthology.orchestra.run.smithi195.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-16T20:31:32.774 INFO:teuthology.orchestra.run.smithi195.stdout: python2.7-minimal 2024-03-16T20:31:32.775 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-16T20:31:32.776 INFO:teuthology.orchestra.run.smithi195.stdout:The following additional packages will be installed: 2024-03-16T20:31:32.776 INFO:teuthology.orchestra.run.smithi195.stdout: linux-generic linux-headers-5.15.0-100 linux-headers-5.15.0-100-generic 2024-03-16T20:31:32.776 INFO:teuthology.orchestra.run.smithi195.stdout: linux-headers-generic linux-image-5.15.0-100-generic 2024-03-16T20:31:32.776 INFO:teuthology.orchestra.run.smithi195.stdout: linux-modules-5.15.0-100-generic linux-modules-extra-5.15.0-100-generic 2024-03-16T20:31:32.777 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2024-03-16T20:31:32.777 INFO:teuthology.orchestra.run.smithi195.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-16T20:31:32.777 INFO:teuthology.orchestra.run.smithi195.stdout:Recommended packages: 2024-03-16T20:31:32.778 INFO:teuthology.orchestra.run.smithi195.stdout: thermald 2024-03-16T20:31:32.807 INFO:teuthology.orchestra.run.smithi122.stdout:The following NEW packages will be installed: 2024-03-16T20:31:32.807 INFO:teuthology.orchestra.run.smithi122.stdout: linux-headers-5.15.0-100 linux-headers-5.15.0-100-generic 2024-03-16T20:31:32.807 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.15.0-100-generic linux-modules-5.15.0-100-generic 2024-03-16T20:31:32.807 INFO:teuthology.orchestra.run.smithi122.stdout: linux-modules-extra-5.15.0-100-generic 2024-03-16T20:31:32.809 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages will be upgraded: 2024-03-16T20:31:32.809 INFO:teuthology.orchestra.run.smithi122.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-16T20:31:32.820 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2024-03-16T20:31:32.820 INFO:teuthology.orchestra.run.smithi195.stdout: linux-headers-5.15.0-100 linux-headers-5.15.0-100-generic 2024-03-16T20:31:32.820 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.15.0-100-generic linux-modules-5.15.0-100-generic 2024-03-16T20:31:32.821 INFO:teuthology.orchestra.run.smithi195.stdout: linux-modules-extra-5.15.0-100-generic 2024-03-16T20:31:32.822 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages will be upgraded: 2024-03-16T20:31:32.822 INFO:teuthology.orchestra.run.smithi195.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-16T20:31:32.878 INFO:teuthology.orchestra.run.smithi195.stdout:3 upgraded, 5 newly installed, 0 to remove and 297 not upgraded. 2024-03-16T20:31:32.878 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 113 MB of archives. 2024-03-16T20:31:32.878 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-16T20:31:32.878 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-100-generic amd64 5.15.0-100.110 [22.6 MB] 2024-03-16T20:31:32.988 INFO:teuthology.orchestra.run.smithi122.stdout:3 upgraded, 5 newly installed, 0 to remove and 297 not upgraded. 2024-03-16T20:31:32.988 INFO:teuthology.orchestra.run.smithi122.stdout:Need to get 113 MB of archives. 2024-03-16T20:31:32.988 INFO:teuthology.orchestra.run.smithi122.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-16T20:31:32.988 INFO:teuthology.orchestra.run.smithi122.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-100-generic amd64 5.15.0-100.110 [22.6 MB] 2024-03-16T20:31:33.301 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-100-generic amd64 5.15.0-100.110 [11.5 MB] 2024-03-16T20:31:33.407 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-100-generic amd64 5.15.0-100.110 [63.8 MB] 2024-03-16T20:31:34.328 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.100.97 [1,696 B] 2024-03-16T20:31:34.328 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.100.97 [2,454 B] 2024-03-16T20:31:34.328 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-100 all 5.15.0-100.110 [12.3 MB] 2024-03-16T20:31:34.392 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-100-generic amd64 5.15.0-100.110 [11.5 MB] 2024-03-16T20:31:34.555 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-100-generic amd64 5.15.0-100.110 [2,858 kB] 2024-03-16T20:31:34.579 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.100.97 [2,292 B] 2024-03-16T20:31:34.744 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-100-generic amd64 5.15.0-100.110 [63.8 MB] 2024-03-16T20:31:34.966 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 113 MB in 2s (64.6 MB/s) 2024-03-16T20:31:35.148 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-modules-5.15.0-100-generic. 2024-03-16T20:31:37.031 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.100.97 [1,696 B] 2024-03-16T20:31:37.031 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.100.97 [2,454 B] 2024-03-16T20:31:37.031 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-100 all 5.15.0-100.110 [12.3 MB] 2024-03-16T20:31:37.200 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-16T20:31:37.204 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../0-linux-modules-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-16T20:31:37.296 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-modules-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:31:37.976 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-100-generic amd64 5.15.0-100.110 [2,858 kB] 2024-03-16T20:31:38.055 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.100.97 [2,292 B] 2024-03-16T20:31:38.450 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 113 MB in 5s (21.6 MB/s) 2024-03-16T20:31:38.652 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-modules-5.15.0-100-generic. 2024-03-16T20:31:39.722 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-image-5.15.0-100-generic. 2024-03-16T20:31:39.744 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../1-linux-image-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-16T20:31:39.845 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:31:40.309 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-100-generic. 2024-03-16T20:31:40.329 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-16T20:31:40.366 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-modules-extra-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:31:40.695 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-16T20:31:40.699 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../0-linux-modules-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-16T20:31:40.807 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-modules-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:31:42.976 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-image-5.15.0-100-generic. 2024-03-16T20:31:42.997 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../1-linux-image-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-16T20:31:43.094 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:31:43.604 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-100-generic. 2024-03-16T20:31:43.624 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-16T20:31:43.662 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-modules-extra-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:31:47.426 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../3-linux-generic_5.15.0.100.97_amd64.deb ... 2024-03-16T20:31:47.553 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-16T20:31:47.911 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.100.97_amd64.deb ... 2024-03-16T20:31:48.039 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-image-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-16T20:31:48.359 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-headers-5.15.0-100. 2024-03-16T20:31:48.382 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../5-linux-headers-5.15.0-100_5.15.0-100.110_all.deb ... 2024-03-16T20:31:48.417 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-5.15.0-100 (5.15.0-100.110) ... 2024-03-16T20:31:50.754 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../3-linux-generic_5.15.0.100.97_amd64.deb ... 2024-03-16T20:31:50.882 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-16T20:31:51.223 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.100.97_amd64.deb ... 2024-03-16T20:31:51.352 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-image-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-16T20:31:51.638 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-headers-5.15.0-100. 2024-03-16T20:31:51.660 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../5-linux-headers-5.15.0-100_5.15.0-100.110_all.deb ... 2024-03-16T20:31:51.698 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-5.15.0-100 (5.15.0-100.110) ... 2024-03-16T20:31:54.054 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package linux-headers-5.15.0-100-generic. 2024-03-16T20:31:54.091 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../6-linux-headers-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-16T20:31:54.137 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:31:57.377 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-headers-5.15.0-100-generic. 2024-03-16T20:31:57.414 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../6-linux-headers-5.15.0-100-generic_5.15.0-100.110_amd64.deb ... 2024-03-16T20:31:57.439 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.100.97_amd64.deb ... 2024-03-16T20:31:57.452 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:31:57.565 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking linux-headers-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-16T20:31:57.915 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-5.15.0-100 (5.15.0-100.110) ... 2024-03-16T20:31:58.061 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:31:58.188 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-headers-generic (5.15.0.100.97) ... 2024-03-16T20:31:58.316 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:31:59.632 INFO:teuthology.orchestra.run.smithi195.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-16T20:31:59.632 INFO:teuthology.orchestra.run.smithi195.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-16T20:31:59.632 INFO:teuthology.orchestra.run.smithi195.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-100-generic 2024-03-16T20:31:59.632 INFO:teuthology.orchestra.run.smithi195.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-100-generic 2024-03-16T20:31:59.785 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-modules-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:32:00.158 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.100.97_amd64.deb ... 2024-03-16T20:32:01.073 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-generic (5.15.0.100.97) over (5.15.0.56.54) ... 2024-03-16T20:32:01.403 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-modules-extra-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:32:01.423 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-5.15.0-100 (5.15.0-100.110) ... 2024-03-16T20:32:01.569 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:32:01.695 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-generic (5.15.0.100.97) ... 2024-03-16T20:32:01.824 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:32:02.788 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-image-generic (5.15.0.100.97) ... 2024-03-16T20:32:02.914 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up linux-generic (5.15.0.100.97) ... 2024-03-16T20:32:03.040 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:32:03.101 INFO:teuthology.orchestra.run.smithi195.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-16T20:32:03.101 INFO:teuthology.orchestra.run.smithi195.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-100-generic 2024-03-16T20:32:03.183 INFO:teuthology.orchestra.run.smithi122.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-16T20:32:03.183 INFO:teuthology.orchestra.run.smithi122.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-16T20:32:03.183 INFO:teuthology.orchestra.run.smithi122.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-100-generic 2024-03-16T20:32:03.183 INFO:teuthology.orchestra.run.smithi122.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-100-generic 2024-03-16T20:32:03.368 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-modules-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:32:05.013 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-modules-extra-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:32:07.271 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-image-generic (5.15.0.100.97) ... 2024-03-16T20:32:07.413 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-generic (5.15.0.100.97) ... 2024-03-16T20:32:07.540 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for linux-image-5.15.0-100-generic (5.15.0-100.110) ... 2024-03-16T20:32:07.606 INFO:teuthology.orchestra.run.smithi122.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-16T20:32:07.606 INFO:teuthology.orchestra.run.smithi122.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-100-generic 2024-03-16T20:32:17.262 INFO:teuthology.orchestra.run.smithi195.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-16T20:32:17.263 INFO:teuthology.orchestra.run.smithi195.stdout:Sourcing file `/etc/default/grub' 2024-03-16T20:32:17.282 INFO:teuthology.orchestra.run.smithi195.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-16T20:32:17.295 INFO:teuthology.orchestra.run.smithi195.stdout:Generating grub configuration file ... 2024-03-16T20:32:17.575 INFO:teuthology.orchestra.run.smithi195.stdout:Found linux image: /boot/vmlinuz-5.15.0-100-generic 2024-03-16T20:32:17.605 INFO:teuthology.orchestra.run.smithi195.stdout:Found initrd image: /boot/initrd.img-5.15.0-100-generic 2024-03-16T20:32:17.906 INFO:teuthology.orchestra.run.smithi195.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-16T20:32:17.912 INFO:teuthology.orchestra.run.smithi195.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-16T20:32:17.976 INFO:teuthology.orchestra.run.smithi195.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-16T20:32:17.982 INFO:teuthology.orchestra.run.smithi195.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-16T20:32:18.181 INFO:teuthology.orchestra.run.smithi195.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-16T20:32:18.181 INFO:teuthology.orchestra.run.smithi195.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-16T20:32:18.181 INFO:teuthology.orchestra.run.smithi195.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-16T20:32:18.226 INFO:teuthology.orchestra.run.smithi195.stdout:done 2024-03-16T20:32:18.644 DEBUG:teuthology.orchestra.run.smithi195:> dpkg -s linux-image-generic 2024-03-16T20:32:18.661 INFO:teuthology.orchestra.run.smithi195.stdout:Package: linux-image-generic 2024-03-16T20:32:18.661 INFO:teuthology.orchestra.run.smithi195.stdout:Status: install ok installed 2024-03-16T20:32:18.661 INFO:teuthology.orchestra.run.smithi195.stdout:Priority: optional 2024-03-16T20:32:18.661 INFO:teuthology.orchestra.run.smithi195.stdout:Section: kernel 2024-03-16T20:32:18.661 INFO:teuthology.orchestra.run.smithi195.stdout:Installed-Size: 21 2024-03-16T20:32:18.661 INFO:teuthology.orchestra.run.smithi195.stdout:Maintainer: Ubuntu Kernel Team 2024-03-16T20:32:18.661 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: amd64 2024-03-16T20:32:18.661 INFO:teuthology.orchestra.run.smithi195.stdout:Source: linux-meta 2024-03-16T20:32:18.661 INFO:teuthology.orchestra.run.smithi195.stdout:Version: 5.15.0.100.97 2024-03-16T20:32:18.661 INFO:teuthology.orchestra.run.smithi195.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-100), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-16T20:32:18.661 INFO:teuthology.orchestra.run.smithi195.stdout:Depends: linux-image-5.15.0-100-generic, linux-modules-extra-5.15.0-100-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-16T20:32:18.661 INFO:teuthology.orchestra.run.smithi195.stdout:Recommends: thermald 2024-03-16T20:32:18.662 INFO:teuthology.orchestra.run.smithi195.stdout:Description: Generic Linux kernel image 2024-03-16T20:32:18.662 INFO:teuthology.orchestra.run.smithi195.stdout: This package will always depend on the latest generic kernel image 2024-03-16T20:32:18.662 INFO:teuthology.orchestra.run.smithi195.stdout: available. 2024-03-16T20:32:18.662 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-100-generic 2024-03-16T20:32:18.662 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-16T20:32:18.662 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-16T20:32:18.662 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-16T20:32:18.662 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi195.front.sepia.ceph.com, path=None, version=distro) 2024-03-16T20:32:18.662 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get clean 2024-03-16T20:32:18.833 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2024-03-16T20:32:18.989 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-16T20:32:19.085 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-16T20:32:19.172 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-16T20:32:19.249 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-16T20:32:20.354 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-03-16T20:32:20.371 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-16T20:32:20.432 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-03-16T20:32:20.632 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-03-16T20:32:20.632 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-03-16T20:32:20.792 INFO:teuthology.orchestra.run.smithi195.stdout:linux-image-generic is already the newest version (5.15.0.100.97). 2024-03-16T20:32:20.792 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2024-03-16T20:32:20.792 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-16T20:32:20.792 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-16T20:32:20.792 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-16T20:32:20.793 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-16T20:32:20.793 INFO:teuthology.orchestra.run.smithi195.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-16T20:32:20.793 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-16T20:32:20.830 INFO:teuthology.orchestra.run.smithi195.stdout:0 upgraded, 0 newly installed, 0 to remove and 297 not upgraded. 2024-03-16T20:32:20.832 DEBUG:teuthology.orchestra.run.smithi195:> dpkg -s linux-image-generic 2024-03-16T20:32:20.847 INFO:teuthology.orchestra.run.smithi195.stdout:Package: linux-image-generic 2024-03-16T20:32:20.847 INFO:teuthology.orchestra.run.smithi195.stdout:Status: install ok installed 2024-03-16T20:32:20.847 INFO:teuthology.orchestra.run.smithi195.stdout:Priority: optional 2024-03-16T20:32:20.847 INFO:teuthology.orchestra.run.smithi195.stdout:Section: kernel 2024-03-16T20:32:20.847 INFO:teuthology.orchestra.run.smithi195.stdout:Installed-Size: 21 2024-03-16T20:32:20.847 INFO:teuthology.orchestra.run.smithi195.stdout:Maintainer: Ubuntu Kernel Team 2024-03-16T20:32:20.847 INFO:teuthology.orchestra.run.smithi195.stdout:Architecture: amd64 2024-03-16T20:32:20.848 INFO:teuthology.orchestra.run.smithi195.stdout:Source: linux-meta 2024-03-16T20:32:20.848 INFO:teuthology.orchestra.run.smithi195.stdout:Version: 5.15.0.100.97 2024-03-16T20:32:20.848 INFO:teuthology.orchestra.run.smithi195.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-100), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-16T20:32:20.848 INFO:teuthology.orchestra.run.smithi195.stdout:Depends: linux-image-5.15.0-100-generic, linux-modules-extra-5.15.0-100-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-16T20:32:20.848 INFO:teuthology.orchestra.run.smithi195.stdout:Recommends: thermald 2024-03-16T20:32:20.848 INFO:teuthology.orchestra.run.smithi195.stdout:Description: Generic Linux kernel image 2024-03-16T20:32:20.848 INFO:teuthology.orchestra.run.smithi195.stdout: This package will always depend on the latest generic kernel image 2024-03-16T20:32:20.848 INFO:teuthology.orchestra.run.smithi195.stdout: available. 2024-03-16T20:32:20.848 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-03-16T20:32:20.892 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.WEE9AwET9t 2024-03-16T20:32:20.892 DEBUG:teuthology.orchestra.run.smithi195:> sudo cp /boot/grub/grub.cfg /tmp/tmp.WEE9AwET9t 2024-03-16T20:32:20.949 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.WEE9AwET9t 2024-03-16T20:32:21.058 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.WEE9AwET9t is 10KB 2024-03-16T20:32:21.071 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.WEE9AwET9t 2024-03-16T20:32:21.076 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-16T20:32:21.131 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:32:21.131 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-16T20:32:21.131 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-16T20:32:21.205 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-100-generic 2024-03-16T20:32:21.205 DEBUG:teuthology.orchestra.run.smithi195:> sudo update-grub 2024-03-16T20:32:21.744 INFO:teuthology.orchestra.run.smithi122.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-16T20:32:21.745 INFO:teuthology.orchestra.run.smithi122.stdout:Sourcing file `/etc/default/grub' 2024-03-16T20:32:21.761 INFO:teuthology.orchestra.run.smithi122.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-16T20:32:21.782 INFO:teuthology.orchestra.run.smithi122.stdout:Generating grub configuration file ... 2024-03-16T20:32:22.090 INFO:teuthology.orchestra.run.smithi122.stdout:Found linux image: /boot/vmlinuz-5.15.0-100-generic 2024-03-16T20:32:22.120 INFO:teuthology.orchestra.run.smithi122.stdout:Found initrd image: /boot/initrd.img-5.15.0-100-generic 2024-03-16T20:32:22.432 INFO:teuthology.orchestra.run.smithi122.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-16T20:32:22.437 INFO:teuthology.orchestra.run.smithi122.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-16T20:32:22.489 INFO:teuthology.orchestra.run.smithi195.stderr:Sourcing file `/etc/default/grub' 2024-03-16T20:32:22.490 INFO:teuthology.orchestra.run.smithi195.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-16T20:32:22.492 INFO:teuthology.orchestra.run.smithi195.stderr:Generating grub configuration file ... 2024-03-16T20:32:22.508 INFO:teuthology.orchestra.run.smithi122.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-16T20:32:22.513 INFO:teuthology.orchestra.run.smithi122.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-16T20:32:22.686 INFO:teuthology.orchestra.run.smithi195.stderr:Found linux image: /boot/vmlinuz-5.15.0-100-generic 2024-03-16T20:32:22.691 INFO:teuthology.orchestra.run.smithi122.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-16T20:32:22.691 INFO:teuthology.orchestra.run.smithi122.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-16T20:32:22.691 INFO:teuthology.orchestra.run.smithi122.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-16T20:32:22.692 INFO:teuthology.orchestra.run.smithi195.stderr:Found initrd image: /boot/initrd.img-5.15.0-100-generic 2024-03-16T20:32:22.731 INFO:teuthology.orchestra.run.smithi122.stdout:done 2024-03-16T20:32:22.939 INFO:teuthology.orchestra.run.smithi195.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-16T20:32:22.944 INFO:teuthology.orchestra.run.smithi195.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-16T20:32:22.983 INFO:teuthology.orchestra.run.smithi195.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-16T20:32:22.989 INFO:teuthology.orchestra.run.smithi195.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-16T20:32:23.108 INFO:teuthology.orchestra.run.smithi195.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-16T20:32:23.108 INFO:teuthology.orchestra.run.smithi195.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-16T20:32:23.108 INFO:teuthology.orchestra.run.smithi195.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-16T20:32:23.123 INFO:teuthology.orchestra.run.smithi195.stderr:done 2024-03-16T20:32:23.125 DEBUG:teuthology.orchestra.run.smithi195:> sudo shutdown -r now 2024-03-16T20:32:23.186 DEBUG:teuthology.orchestra.run.smithi122:> dpkg -s linux-image-generic 2024-03-16T20:32:23.206 INFO:teuthology.orchestra.run.smithi122.stdout:Package: linux-image-generic 2024-03-16T20:32:23.206 INFO:teuthology.orchestra.run.smithi122.stdout:Status: install ok installed 2024-03-16T20:32:23.206 INFO:teuthology.orchestra.run.smithi122.stdout:Priority: optional 2024-03-16T20:32:23.206 INFO:teuthology.orchestra.run.smithi122.stdout:Section: kernel 2024-03-16T20:32:23.207 INFO:teuthology.orchestra.run.smithi122.stdout:Installed-Size: 21 2024-03-16T20:32:23.207 INFO:teuthology.orchestra.run.smithi122.stdout:Maintainer: Ubuntu Kernel Team 2024-03-16T20:32:23.207 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: amd64 2024-03-16T20:32:23.207 INFO:teuthology.orchestra.run.smithi122.stdout:Source: linux-meta 2024-03-16T20:32:23.207 INFO:teuthology.orchestra.run.smithi122.stdout:Version: 5.15.0.100.97 2024-03-16T20:32:23.207 INFO:teuthology.orchestra.run.smithi122.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-100), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-16T20:32:23.207 INFO:teuthology.orchestra.run.smithi122.stdout:Depends: linux-image-5.15.0-100-generic, linux-modules-extra-5.15.0-100-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-16T20:32:23.207 INFO:teuthology.orchestra.run.smithi122.stdout:Recommends: thermald 2024-03-16T20:32:23.207 INFO:teuthology.orchestra.run.smithi122.stdout:Description: Generic Linux kernel image 2024-03-16T20:32:23.207 INFO:teuthology.orchestra.run.smithi122.stdout: This package will always depend on the latest generic kernel image 2024-03-16T20:32:23.207 INFO:teuthology.orchestra.run.smithi122.stdout: available. 2024-03-16T20:32:23.207 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-100-generic 2024-03-16T20:32:23.207 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-16T20:32:23.208 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-16T20:32:23.208 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-16T20:32:23.208 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi122.front.sepia.ceph.com, path=None, version=distro) 2024-03-16T20:32:23.208 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get clean 2024-03-16T20:32:23.394 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-03-16T20:32:23.530 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-16T20:32:23.643 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-16T20:32:23.726 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-16T20:32:23.806 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-16T20:32:24.907 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-03-16T20:32:24.925 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-16T20:32:24.974 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-03-16T20:32:25.158 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-03-16T20:32:25.159 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-03-16T20:32:25.310 INFO:teuthology.orchestra.run.smithi122.stdout:linux-image-generic is already the newest version (5.15.0.100.97). 2024-03-16T20:32:25.310 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-03-16T20:32:25.310 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-16T20:32:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-16T20:32:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-16T20:32:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-16T20:32:25.311 INFO:teuthology.orchestra.run.smithi122.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-16T20:32:25.311 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-16T20:32:25.348 INFO:teuthology.orchestra.run.smithi122.stdout:0 upgraded, 0 newly installed, 0 to remove and 297 not upgraded. 2024-03-16T20:32:25.350 DEBUG:teuthology.orchestra.run.smithi122:> dpkg -s linux-image-generic 2024-03-16T20:32:25.407 INFO:teuthology.orchestra.run.smithi122.stdout:Package: linux-image-generic 2024-03-16T20:32:25.408 INFO:teuthology.orchestra.run.smithi122.stdout:Status: install ok installed 2024-03-16T20:32:25.408 INFO:teuthology.orchestra.run.smithi122.stdout:Priority: optional 2024-03-16T20:32:25.408 INFO:teuthology.orchestra.run.smithi122.stdout:Section: kernel 2024-03-16T20:32:25.408 INFO:teuthology.orchestra.run.smithi122.stdout:Installed-Size: 21 2024-03-16T20:32:25.408 INFO:teuthology.orchestra.run.smithi122.stdout:Maintainer: Ubuntu Kernel Team 2024-03-16T20:32:25.408 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: amd64 2024-03-16T20:32:25.408 INFO:teuthology.orchestra.run.smithi122.stdout:Source: linux-meta 2024-03-16T20:32:25.408 INFO:teuthology.orchestra.run.smithi122.stdout:Version: 5.15.0.100.97 2024-03-16T20:32:25.408 INFO:teuthology.orchestra.run.smithi122.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-100), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-16T20:32:25.408 INFO:teuthology.orchestra.run.smithi122.stdout:Depends: linux-image-5.15.0-100-generic, linux-modules-extra-5.15.0-100-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-16T20:32:25.408 INFO:teuthology.orchestra.run.smithi122.stdout:Recommends: thermald 2024-03-16T20:32:25.408 INFO:teuthology.orchestra.run.smithi122.stdout:Description: Generic Linux kernel image 2024-03-16T20:32:25.409 INFO:teuthology.orchestra.run.smithi122.stdout: This package will always depend on the latest generic kernel image 2024-03-16T20:32:25.409 INFO:teuthology.orchestra.run.smithi122.stdout: available. 2024-03-16T20:32:25.409 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-03-16T20:32:25.453 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.bqJnYrx6mc 2024-03-16T20:32:25.453 DEBUG:teuthology.orchestra.run.smithi122:> sudo cp /boot/grub/grub.cfg /tmp/tmp.bqJnYrx6mc 2024-03-16T20:32:25.509 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.bqJnYrx6mc 2024-03-16T20:32:25.623 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.bqJnYrx6mc is 10KB 2024-03-16T20:32:25.636 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.bqJnYrx6mc 2024-03-16T20:32:25.642 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-16T20:32:25.698 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:32:25.698 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-16T20:32:25.698 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-16T20:32:25.771 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-100-generic 2024-03-16T20:32:25.771 DEBUG:teuthology.orchestra.run.smithi122:> sudo update-grub 2024-03-16T20:32:27.065 INFO:teuthology.orchestra.run.smithi122.stderr:Sourcing file `/etc/default/grub' 2024-03-16T20:32:27.067 INFO:teuthology.orchestra.run.smithi122.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-16T20:32:27.068 INFO:teuthology.orchestra.run.smithi122.stderr:Generating grub configuration file ... 2024-03-16T20:32:27.287 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.15.0-100-generic 2024-03-16T20:32:27.292 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.15.0-100-generic 2024-03-16T20:32:27.513 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-16T20:32:27.519 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-16T20:32:27.558 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-16T20:32:27.563 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-16T20:32:27.695 INFO:teuthology.orchestra.run.smithi122.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-16T20:32:27.697 INFO:teuthology.orchestra.run.smithi122.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-16T20:32:27.697 INFO:teuthology.orchestra.run.smithi122.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-16T20:32:27.706 INFO:teuthology.orchestra.run.smithi122.stderr:done 2024-03-16T20:32:27.710 DEBUG:teuthology.orchestra.run.smithi122:> sudo shutdown -r now 2024-03-16T20:32:53.146 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-16T20:32:53.146 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-03-16T20:32:53.147 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T20:32:57.714 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-16T20:32:57.714 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-03-16T20:32:57.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T20:33:53.199 DEBUG:teuthology.orchestra.remote:timed out 2024-03-16T20:33:57.722 DEBUG:teuthology.orchestra.remote:timed out 2024-03-16T20:34:02.201 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-03-16T20:34:02.202 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T20:34:02.204 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.195 2024-03-16T20:34:06.727 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-03-16T20:34:06.728 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T20:34:07.529 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-03-16T20:34:08.577 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-03-16T20:34:08.577 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-100-generic"... 2024-03-16T20:34:08.578 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-03-16T20:34:08.624 INFO:teuthology.orchestra.run.smithi122.stdout:5.15.0-100-generic 2024-03-16T20:34:08.625 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-100-generic vs 5.15.0-100-generic 2024-03-16T20:34:08.625 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-16T20:34:08.625 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-16T20:34:09.626 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-16T20:34:09.626 DEBUG:teuthology.orchestra.run.smithi122:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-16T20:34:10.078 INFO:teuthology.orchestra.run.smithi122.stdout:ttyS1 2024-03-16T20:34:10.101 DEBUG:teuthology.parallel:result is None 2024-03-16T20:34:14.206 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-03-16T20:34:14.206 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T20:34:14.487 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-03-16T20:34:15.052 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-03-16T20:34:15.052 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-100-generic"... 2024-03-16T20:34:15.052 DEBUG:teuthology.orchestra.run.smithi195:> uname -r 2024-03-16T20:34:15.100 INFO:teuthology.orchestra.run.smithi195.stdout:5.15.0-100-generic 2024-03-16T20:34:15.100 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-100-generic vs 5.15.0-100-generic 2024-03-16T20:34:15.100 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-16T20:34:15.100 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-16T20:34:16.101 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-16T20:34:16.102 DEBUG:teuthology.orchestra.run.smithi195:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-16T20:34:16.179 INFO:teuthology.orchestra.run.smithi195.stdout:ttyS1 2024-03-16T20:34:16.202 DEBUG:teuthology.parallel:result is None 2024-03-16T20:34:16.202 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-16T20:34:16.209 INFO:teuthology.task.internal:Creating test directory... 2024-03-16T20:34:16.209 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T20:34:16.212 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-16T20:34:16.217 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-16T20:34:16.231 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-16T20:34:16.239 INFO:teuthology.task.internal:Creating archive directory... 2024-03-16T20:34:16.239 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T20:34:16.260 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-16T20:34:16.370 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-16T20:34:16.378 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-16T20:34:16.378 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T20:34:16.381 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-16T20:34:16.398 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T20:34:16.406 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T20:34:16.407 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T20:34:16.415 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-16T20:34:16.417 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-16T20:34:16.424 INFO:teuthology.task.internal:Configuring sudo... 2024-03-16T20:34:16.424 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T20:34:16.460 DEBUG:teuthology.orchestra.run.smithi195:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-16T20:34:16.478 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-16T20:34:16.486 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-16T20:34:16.487 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T20:34:16.520 DEBUG:teuthology.orchestra.run.smithi195:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-16T20:34:16.525 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T20:34:16.572 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T20:34:16.620 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:34:16.620 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T20:34:16.685 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-16T20:34:16.691 DEBUG:teuthology.orchestra.run.smithi195:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-16T20:34:16.736 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:34:16.737 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-16T20:34:16.801 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-03-16T20:34:16.803 DEBUG:teuthology.orchestra.run.smithi195:> sudo service rsyslog restart 2024-03-16T20:34:16.874 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-16T20:34:16.881 INFO:teuthology.task.internal:Starting timer... 2024-03-16T20:34:16.882 INFO:teuthology.run_tasks:Running task pcp... 2024-03-16T20:34:16.892 INFO:teuthology.run_tasks:Running task selinux... 2024-03-16T20:34:16.901 DEBUG:teuthology.task.selinux:Excluding smithi122: OS 'ubuntu' does not support SELinux 2024-03-16T20:34:16.901 DEBUG:teuthology.task.selinux:Excluding smithi195: OS 'ubuntu' does not support SELinux 2024-03-16T20:34:16.901 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-16T20:34:16.901 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-03-16T20:34:16.901 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-16T20:34:16.902 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-16T20:34:16.911 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-16T20:34:16.927 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-16T20:34:16.928 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi122.front.sepia.ceph.com,smithi195.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-16T20:41:15.566 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi122.front.sepia.ceph.com'), Remote(name='ubuntu@smithi195.front.sepia.ceph.com')] 2024-03-16T20:41:15.567 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-03-16T20:41:15.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T20:41:15.646 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-03-16T20:41:15.722 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-03-16T20:41:15.723 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-03-16T20:41:15.723 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi195.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-16T20:41:15.800 DEBUG:teuthology.orchestra.run.smithi195:> true 2024-03-16T20:41:15.878 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi195.front.sepia.ceph.com' 2024-03-16T20:41:15.879 INFO:teuthology.run_tasks:Running task clock... 2024-03-16T20:41:15.889 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-16T20:41:15.889 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T20:41:15.889 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T20:41:15.891 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-16T20:41:15.892 DEBUG:teuthology.orchestra.run.smithi195:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T20:41:15.917 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-16T20:41:15.917 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: Command line: ntpd -gq 2024-03-16T20:41:15.917 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: ---------------------------------------------------- 2024-03-16T20:41:15.917 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: ntp-4 is maintained by Network Time Foundation, 2024-03-16T20:41:15.917 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-16T20:41:15.917 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: corporation. Support and training for ntp-4 are 2024-03-16T20:41:15.917 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: available at https://www.nwtime.org/support 2024-03-16T20:41:15.917 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: ---------------------------------------------------- 2024-03-16T20:41:15.917 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: proto: precision = 0.044 usec (-24) 2024-03-16T20:41:15.918 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: basedate set to 2022-02-04 2024-03-16T20:41:15.918 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: gps base set to 2022-02-06 (week 2196) 2024-03-16T20:41:15.918 INFO:teuthology.orchestra.run.smithi122.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-16T20:41:15.919 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-16T20:41:15.919 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: restrict ::: KOD does nothing without LIMITED. 2024-03-16T20:41:15.919 INFO:teuthology.orchestra.run.smithi122.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-16T20:41:15.920 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: Listen and drop on 0 v6wildcard [::]:123 2024-03-16T20:41:15.920 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-16T20:41:15.920 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: Listen normally on 2 lo 127.0.0.1:123 2024-03-16T20:41:15.920 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: Listen normally on 3 enp3s0f1 172.21.15.122:123 2024-03-16T20:41:15.920 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: Listen normally on 4 lo [::1]:123 2024-03-16T20:41:15.920 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d515%5]:123 2024-03-16T20:41:15.920 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:15 ntpd[17177]: Listening on routing socket on fd #22 for interface updates 2024-03-16T20:41:15.943 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-16T20:41:15.943 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: Command line: ntpd -gq 2024-03-16T20:41:15.943 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: ---------------------------------------------------- 2024-03-16T20:41:15.943 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: ntp-4 is maintained by Network Time Foundation, 2024-03-16T20:41:15.943 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-16T20:41:15.943 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: corporation. Support and training for ntp-4 are 2024-03-16T20:41:15.943 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: available at https://www.nwtime.org/support 2024-03-16T20:41:15.944 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: ---------------------------------------------------- 2024-03-16T20:41:15.944 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: proto: precision = 0.045 usec (-24) 2024-03-16T20:41:15.944 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: basedate set to 2022-02-04 2024-03-16T20:41:15.944 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: gps base set to 2022-02-06 (week 2196) 2024-03-16T20:41:15.945 INFO:teuthology.orchestra.run.smithi195.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-16T20:41:15.945 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-16T20:41:15.945 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: restrict ::: KOD does nothing without LIMITED. 2024-03-16T20:41:15.945 INFO:teuthology.orchestra.run.smithi195.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-16T20:41:15.946 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: Listen and drop on 0 v6wildcard [::]:123 2024-03-16T20:41:15.946 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-16T20:41:15.946 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: Listen normally on 2 lo 127.0.0.1:123 2024-03-16T20:41:15.946 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: Listen normally on 3 enp3s0f1 172.21.15.195:123 2024-03-16T20:41:15.946 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: Listen normally on 4 lo [::1]:123 2024-03-16T20:41:15.947 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:8081%5]:123 2024-03-16T20:41:15.947 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:15 ntpd[17182]: Listening on routing socket on fd #22 for interface updates 2024-03-16T20:41:16.917 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:16 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:16.917 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:16 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:16.918 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:16 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:16.918 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:16 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:16.918 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:16 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:16.918 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:16 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:16.943 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:16 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:16.943 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:16 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:17.943 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:17 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:17.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:17 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:17.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:17 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:17.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:17 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:17.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:17 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:17.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:17 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:18.918 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:18 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:18.918 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:18 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:18.918 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:18 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:18.918 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:18 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:18.918 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:18 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:18.918 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:18 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:18.918 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:18 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:18.918 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:18 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:18.918 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:18 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:18.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:18 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:18.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:18 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:18.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:18 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:18.943 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:18 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:18.943 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:18 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:18.943 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:18 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:18.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:18 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:18.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:18 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:18.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:18 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:19.917 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:19 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:19.917 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:19 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:19.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:19 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:19.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:19 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:19.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:19 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:19.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:19 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:19.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:19 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:19.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:19 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:19.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:19 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:19.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:19 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:19.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:19 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:19.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:19 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:19.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:19 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:19.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:19 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:20.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:20.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:20.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:20.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:20.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:20.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:20.920 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:20.920 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:20.920 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:20.920 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:20.920 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:20.920 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:20.920 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:20.920 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:20.920 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:20.920 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:20.921 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:20.921 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:20 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:20.943 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:20 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:20.943 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:20 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:20.943 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:20 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:20.943 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:20 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:20.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:20 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:20.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:20 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:21.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:21 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:21.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:21 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:21.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:21 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:21.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:21 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:21.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:21 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:21.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:21 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:21.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:21 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:21.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:21 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:21.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:21 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:21.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:21 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:21.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:21 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:21.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:21 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:22.918 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:22.918 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:22.918 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:22.918 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:22.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:22.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:22.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:22.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:22.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:22.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:22.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:22.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:22.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:22.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:22.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:22.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:22.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:22.919 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:22.920 INFO:teuthology.orchestra.run.smithi122.stdout:16 Mar 20:41:22 ntpd[17177]: ntpd: time slew -0.000581 s 2024-03-16T20:41:22.920 INFO:teuthology.orchestra.run.smithi122.stdout:ntpd: time slew -0.000581s 2024-03-16T20:41:22.920 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-16T20:41:22.920 INFO:teuthology.orchestra.run.smithi122.stderr:16 Mar 20:41:22 ntpd[17177]: can't open /var/log/ntpstats/loopstats.20240316: Permission denied 2024-03-16T20:41:22.943 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:22 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:22.943 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:22 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:22.943 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:22 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:22.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:22 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:22.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:22 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:22.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:22 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:22.971 INFO:teuthology.orchestra.run.smithi122.stdout: remote refid st t when poll reach delay offset jitter 2024-03-16T20:41:22.971 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================== 2024-03-16T20:41:22.971 INFO:teuthology.orchestra.run.smithi122.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T20:41:22.971 INFO:teuthology.orchestra.run.smithi122.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T20:41:22.971 INFO:teuthology.orchestra.run.smithi122.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T20:41:22.971 INFO:teuthology.orchestra.run.smithi122.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T20:41:23.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:23 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:23.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:23 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:23.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:23 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:23.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:23 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:23.944 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:23 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:23.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:23 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:23.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:23 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:23.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:23 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:23.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:23 ntpd[17182]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-16T20:41:23.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:23 ntpd[17182]: can't open /var/log/ntpstats/rawstats.20240316: Permission denied 2024-03-16T20:41:23.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:23 ntpd[17182]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-16T20:41:23.945 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:23 ntpd[17182]: can't open /var/log/ntpstats/peerstats.20240316: Permission denied 2024-03-16T20:41:23.945 INFO:teuthology.orchestra.run.smithi195.stdout:16 Mar 20:41:23 ntpd[17182]: ntpd: time slew -0.001211 s 2024-03-16T20:41:23.945 INFO:teuthology.orchestra.run.smithi195.stdout:ntpd: time slew -0.001211s 2024-03-16T20:41:23.946 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:23 ntpd[17182]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-16T20:41:23.946 INFO:teuthology.orchestra.run.smithi195.stderr:16 Mar 20:41:23 ntpd[17182]: can't open /var/log/ntpstats/loopstats.20240316: Permission denied 2024-03-16T20:41:24.040 INFO:teuthology.orchestra.run.smithi195.stdout: remote refid st t when poll reach delay offset jitter 2024-03-16T20:41:24.041 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================== 2024-03-16T20:41:24.041 INFO:teuthology.orchestra.run.smithi195.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T20:41:24.041 INFO:teuthology.orchestra.run.smithi195.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T20:41:24.041 INFO:teuthology.orchestra.run.smithi195.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T20:41:24.041 INFO:teuthology.orchestra.run.smithi195.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T20:41:24.041 INFO:teuthology.run_tasks:Running task install... 2024-03-16T20:41:24.050 DEBUG:teuthology.task.install:project ceph 2024-03-16T20:41:24.050 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b'}} 2024-03-16T20:41:24.051 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b'} 2024-03-16T20:41:24.051 INFO:teuthology.task.install:Using flavor: default 2024-03-16T20:41:24.057 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-16T20:41:24.057 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-16T20:41:24.057 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-key list | grep Ceph 2024-03-16T20:41:24.058 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-key list | grep Ceph 2024-03-16T20:41:24.121 INFO:teuthology.orchestra.run.smithi122.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-16T20:41:24.144 INFO:teuthology.orchestra.run.smithi195.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-16T20:41:24.154 INFO:teuthology.orchestra.run.smithi122.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-16T20:41:24.155 INFO:teuthology.orchestra.run.smithi122.stdout:uid [ unknown] Ceph.com (release key) 2024-03-16T20:41:24.155 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-16T20:41:24.155 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-16T20:41:24.156 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-16T20:41:24.173 INFO:teuthology.orchestra.run.smithi195.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-16T20:41:24.173 INFO:teuthology.orchestra.run.smithi195.stdout:uid [ unknown] Ceph.com (release key) 2024-03-16T20:41:24.173 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-03-16T20:41:24.173 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-03-16T20:41:24.174 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-16T20:41:24.297 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default/ 2024-03-16T20:41:24.297 INFO:teuthology.task.install.deb:Package version is 19.0.0-1681-gaba3c1d2-1jammy 2024-03-16T20:41:24.312 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default/ 2024-03-16T20:41:24.312 INFO:teuthology.task.install.deb:Package version is 19.0.0-1681-gaba3c1d2-1jammy 2024-03-16T20:41:24.423 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:41:24.423 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-16T20:41:24.437 DEBUG:teuthology.orchestra.run.smithi195:> sudo apt-get update 2024-03-16T20:41:24.475 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:41:24.475 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-16T20:41:24.490 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-03-16T20:41:24.614 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-16T20:41:24.626 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-16T20:41:24.645 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-16T20:41:24.656 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-16T20:41:24.656 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-16T20:41:24.670 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-16T20:41:24.689 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-16T20:41:24.706 INFO:teuthology.orchestra.run.smithi122.stdout:Ign:5 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy InRelease 2024-03-16T20:41:24.719 INFO:teuthology.orchestra.run.smithi195.stdout:Ign:4 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy InRelease 2024-03-16T20:41:24.726 INFO:teuthology.orchestra.run.smithi195.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-16T20:41:24.727 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-16T20:41:24.749 INFO:teuthology.orchestra.run.smithi122.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-16T20:41:24.755 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-16T20:41:24.770 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,395 B] 2024-03-16T20:41:24.792 INFO:teuthology.orchestra.run.smithi195.stdout:Ign:7 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-16T20:41:24.797 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-16T20:41:24.829 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-16T20:41:24.871 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,395 B] 2024-03-16T20:41:24.970 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 28.9 kB in 0s (81.2 kB/s) 2024-03-16T20:41:24.995 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 28.9 kB in 0s (86.8 kB/s) 2024-03-16T20:41:25.894 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-03-16T20:41:25.910 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1681-gaba3c1d2-1jammy cephadm=19.0.0-1681-gaba3c1d2-1jammy ceph-mds=19.0.0-1681-gaba3c1d2-1jammy ceph-mgr=19.0.0-1681-gaba3c1d2-1jammy ceph-common=19.0.0-1681-gaba3c1d2-1jammy ceph-fuse=19.0.0-1681-gaba3c1d2-1jammy ceph-test=19.0.0-1681-gaba3c1d2-1jammy ceph-volume=19.0.0-1681-gaba3c1d2-1jammy radosgw=19.0.0-1681-gaba3c1d2-1jammy python3-rados=19.0.0-1681-gaba3c1d2-1jammy python3-rgw=19.0.0-1681-gaba3c1d2-1jammy python3-cephfs=19.0.0-1681-gaba3c1d2-1jammy python3-rbd=19.0.0-1681-gaba3c1d2-1jammy libcephfs2=19.0.0-1681-gaba3c1d2-1jammy libcephfs-dev=19.0.0-1681-gaba3c1d2-1jammy librados2=19.0.0-1681-gaba3c1d2-1jammy librbd1=19.0.0-1681-gaba3c1d2-1jammy rbd-fuse=19.0.0-1681-gaba3c1d2-1jammy python3-cephfs=19.0.0-1681-gaba3c1d2-1jammy cephfs-shell=19.0.0-1681-gaba3c1d2-1jammy cephfs-top=19.0.0-1681-gaba3c1d2-1jammy cephfs-mirror=19.0.0-1681-gaba3c1d2-1jammy 2024-03-16T20:41:25.912 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-03-16T20:41:25.928 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1681-gaba3c1d2-1jammy cephadm=19.0.0-1681-gaba3c1d2-1jammy ceph-mds=19.0.0-1681-gaba3c1d2-1jammy ceph-mgr=19.0.0-1681-gaba3c1d2-1jammy ceph-common=19.0.0-1681-gaba3c1d2-1jammy ceph-fuse=19.0.0-1681-gaba3c1d2-1jammy ceph-test=19.0.0-1681-gaba3c1d2-1jammy ceph-volume=19.0.0-1681-gaba3c1d2-1jammy radosgw=19.0.0-1681-gaba3c1d2-1jammy python3-rados=19.0.0-1681-gaba3c1d2-1jammy python3-rgw=19.0.0-1681-gaba3c1d2-1jammy python3-cephfs=19.0.0-1681-gaba3c1d2-1jammy python3-rbd=19.0.0-1681-gaba3c1d2-1jammy libcephfs2=19.0.0-1681-gaba3c1d2-1jammy libcephfs-dev=19.0.0-1681-gaba3c1d2-1jammy librados2=19.0.0-1681-gaba3c1d2-1jammy librbd1=19.0.0-1681-gaba3c1d2-1jammy rbd-fuse=19.0.0-1681-gaba3c1d2-1jammy python3-cephfs=19.0.0-1681-gaba3c1d2-1jammy cephfs-shell=19.0.0-1681-gaba3c1d2-1jammy cephfs-top=19.0.0-1681-gaba3c1d2-1jammy cephfs-mirror=19.0.0-1681-gaba3c1d2-1jammy 2024-03-16T20:41:25.962 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-03-16T20:41:25.994 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-03-16T20:41:26.133 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-03-16T20:41:26.134 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-03-16T20:41:26.190 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-03-16T20:41:26.191 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-03-16T20:41:26.292 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-03-16T20:41:26.292 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-16T20:41:26.292 INFO:teuthology.orchestra.run.smithi122.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-16T20:41:26.292 INFO:teuthology.orchestra.run.smithi122.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-16T20:41:26.292 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-16T20:41:26.292 INFO:teuthology.orchestra.run.smithi122.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-16T20:41:26.292 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-16T20:41:26.294 INFO:teuthology.orchestra.run.smithi122.stdout:The following additional packages will be installed: 2024-03-16T20:41:26.294 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-16T20:41:26.294 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-16T20:41:26.294 INFO:teuthology.orchestra.run.smithi122.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-16T20:41:26.294 INFO:teuthology.orchestra.run.smithi122.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-16T20:41:26.294 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-16T20:41:26.294 INFO:teuthology.orchestra.run.smithi122.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-16T20:41:26.294 INFO:teuthology.orchestra.run.smithi122.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-16T20:41:26.294 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-16T20:41:26.295 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-16T20:41:26.295 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-16T20:41:26.295 INFO:teuthology.orchestra.run.smithi122.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-16T20:41:26.295 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-16T20:41:26.295 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-16T20:41:26.295 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-16T20:41:26.295 INFO:teuthology.orchestra.run.smithi122.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-16T20:41:26.295 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-16T20:41:26.295 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-16T20:41:26.295 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-16T20:41:26.295 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-16T20:41:26.295 INFO:teuthology.orchestra.run.smithi122.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-16T20:41:26.295 INFO:teuthology.orchestra.run.smithi122.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-16T20:41:26.295 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-16T20:41:26.295 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-16T20:41:26.295 INFO:teuthology.orchestra.run.smithi122.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-03-16T20:41:26.296 INFO:teuthology.orchestra.run.smithi122.stdout:Suggested packages: 2024-03-16T20:41:26.296 INFO:teuthology.orchestra.run.smithi122.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-16T20:41:26.296 INFO:teuthology.orchestra.run.smithi122.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-16T20:41:26.296 INFO:teuthology.orchestra.run.smithi122.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-16T20:41:26.296 INFO:teuthology.orchestra.run.smithi122.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-16T20:41:26.296 INFO:teuthology.orchestra.run.smithi122.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-16T20:41:26.296 INFO:teuthology.orchestra.run.smithi122.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-16T20:41:26.296 INFO:teuthology.orchestra.run.smithi122.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-16T20:41:26.296 INFO:teuthology.orchestra.run.smithi122.stdout:Recommended packages: 2024-03-16T20:41:26.296 INFO:teuthology.orchestra.run.smithi122.stdout: btrfs-tools 2024-03-16T20:41:26.368 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2024-03-16T20:41:26.368 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-16T20:41:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-16T20:41:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-16T20:41:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-16T20:41:26.369 INFO:teuthology.orchestra.run.smithi195.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-16T20:41:26.369 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-16T20:41:26.370 INFO:teuthology.orchestra.run.smithi195.stdout:The following additional packages will be installed: 2024-03-16T20:41:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-16T20:41:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-16T20:41:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-16T20:41:26.370 INFO:teuthology.orchestra.run.smithi195.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-16T20:41:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-16T20:41:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-16T20:41:26.371 INFO:teuthology.orchestra.run.smithi195.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-16T20:41:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-16T20:41:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-16T20:41:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-03-16T20:41:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-03-16T20:41:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-03-16T20:41:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-03-16T20:41:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-03-16T20:41:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-03-16T20:41:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-03-16T20:41:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-03-16T20:41:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-03-16T20:41:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-03-16T20:41:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-03-16T20:41:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-03-16T20:41:26.372 INFO:teuthology.orchestra.run.smithi195.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-03-16T20:41:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-03-16T20:41:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-03-16T20:41:26.373 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2024-03-16T20:41:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-16T20:41:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-16T20:41:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-16T20:41:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-16T20:41:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-16T20:41:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-16T20:41:26.373 INFO:teuthology.orchestra.run.smithi195.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-16T20:41:26.374 INFO:teuthology.orchestra.run.smithi195.stdout:Recommended packages: 2024-03-16T20:41:26.374 INFO:teuthology.orchestra.run.smithi195.stdout: btrfs-tools 2024-03-16T20:41:26.400 INFO:teuthology.orchestra.run.smithi122.stdout:The following NEW packages will be installed: 2024-03-16T20:41:26.400 INFO:teuthology.orchestra.run.smithi122.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-16T20:41:26.400 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-16T20:41:26.400 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-16T20:41:26.400 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-16T20:41:26.400 INFO:teuthology.orchestra.run.smithi122.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-16T20:41:26.400 INFO:teuthology.orchestra.run.smithi122.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-16T20:41:26.400 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-03-16T20:41:26.400 INFO:teuthology.orchestra.run.smithi122.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-16T20:41:26.400 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-16T20:41:26.401 INFO:teuthology.orchestra.run.smithi122.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-16T20:41:26.401 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-16T20:41:26.401 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-16T20:41:26.401 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-16T20:41:26.401 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-16T20:41:26.401 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-16T20:41:26.401 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-16T20:41:26.401 INFO:teuthology.orchestra.run.smithi122.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-16T20:41:26.401 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-16T20:41:26.401 INFO:teuthology.orchestra.run.smithi122.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-16T20:41:26.401 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-16T20:41:26.401 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-16T20:41:26.401 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-16T20:41:26.401 INFO:teuthology.orchestra.run.smithi122.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-16T20:41:26.401 INFO:teuthology.orchestra.run.smithi122.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-16T20:41:26.402 INFO:teuthology.orchestra.run.smithi122.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-16T20:41:26.402 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-16T20:41:26.402 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-16T20:41:26.402 INFO:teuthology.orchestra.run.smithi122.stdout: socat unzip xmlstarlet zip 2024-03-16T20:41:26.462 INFO:teuthology.orchestra.run.smithi122.stdout:0 upgraded, 124 newly installed, 0 to remove and 284 not upgraded. 2024-03-16T20:41:26.462 INFO:teuthology.orchestra.run.smithi122.stdout:Need to get 177 MB of archives. 2024-03-16T20:41:26.463 INFO:teuthology.orchestra.run.smithi122.stdout:After this operation, 735 MB of additional disk space will be used. 2024-03-16T20:41:26.463 INFO:teuthology.orchestra.run.smithi122.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-16T20:41:26.480 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2024-03-16T20:41:26.480 INFO:teuthology.orchestra.run.smithi195.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-16T20:41:26.480 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-16T20:41:26.480 INFO:teuthology.orchestra.run.smithi195.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-03-16T20:41:26.480 INFO:teuthology.orchestra.run.smithi195.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-03-16T20:41:26.481 INFO:teuthology.orchestra.run.smithi195.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-16T20:41:26.481 INFO:teuthology.orchestra.run.smithi195.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-03-16T20:41:26.481 INFO:teuthology.orchestra.run.smithi195.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-03-16T20:41:26.481 INFO:teuthology.orchestra.run.smithi195.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-03-16T20:41:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-03-16T20:41:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-03-16T20:41:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-03-16T20:41:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-03-16T20:41:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-03-16T20:41:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-16T20:41:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-16T20:41:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-16T20:41:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-16T20:41:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-16T20:41:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-16T20:41:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-03-16T20:41:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-03-16T20:41:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-03-16T20:41:26.482 INFO:teuthology.orchestra.run.smithi195.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-16T20:41:26.483 INFO:teuthology.orchestra.run.smithi195.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-16T20:41:26.483 INFO:teuthology.orchestra.run.smithi195.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-16T20:41:26.483 INFO:teuthology.orchestra.run.smithi195.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-16T20:41:26.483 INFO:teuthology.orchestra.run.smithi195.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-03-16T20:41:26.483 INFO:teuthology.orchestra.run.smithi195.stdout: socat unzip xmlstarlet zip 2024-03-16T20:41:26.483 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-16T20:41:26.494 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-16T20:41:26.541 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-16T20:41:26.546 INFO:teuthology.orchestra.run.smithi195.stdout:0 upgraded, 124 newly installed, 0 to remove and 284 not upgraded. 2024-03-16T20:41:26.546 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 177 MB of archives. 2024-03-16T20:41:26.546 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 735 MB of additional disk space will be used. 2024-03-16T20:41:26.546 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-03-16T20:41:26.565 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-16T20:41:26.567 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-16T20:41:26.569 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-16T20:41:26.578 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-03-16T20:41:26.582 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-16T20:41:26.635 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-16T20:41:26.637 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-16T20:41:26.641 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-16T20:41:26.654 INFO:teuthology.orchestra.run.smithi122.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-16T20:41:26.659 INFO:teuthology.orchestra.run.smithi122.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-16T20:41:26.659 INFO:teuthology.orchestra.run.smithi122.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-16T20:41:26.668 INFO:teuthology.orchestra.run.smithi122.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-16T20:41:26.671 INFO:teuthology.orchestra.run.smithi122.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-16T20:41:26.673 INFO:teuthology.orchestra.run.smithi122.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-16T20:41:26.674 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-16T20:41:26.674 INFO:teuthology.orchestra.run.smithi122.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-16T20:41:26.676 INFO:teuthology.orchestra.run.smithi122.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-16T20:41:26.679 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-16T20:41:26.696 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-16T20:41:26.699 INFO:teuthology.orchestra.run.smithi122.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-16T20:41:26.701 INFO:teuthology.orchestra.run.smithi122.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-16T20:41:26.703 INFO:teuthology.orchestra.run.smithi122.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-16T20:41:26.704 INFO:teuthology.orchestra.run.smithi122.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-16T20:41:26.704 INFO:teuthology.orchestra.run.smithi122.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-16T20:41:26.705 INFO:teuthology.orchestra.run.smithi122.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-16T20:41:26.706 INFO:teuthology.orchestra.run.smithi122.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-16T20:41:26.706 INFO:teuthology.orchestra.run.smithi122.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-16T20:41:26.706 INFO:teuthology.orchestra.run.smithi122.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-16T20:41:26.727 INFO:teuthology.orchestra.run.smithi122.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-16T20:41:26.728 INFO:teuthology.orchestra.run.smithi122.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-16T20:41:26.749 INFO:teuthology.orchestra.run.smithi122.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-16T20:41:26.753 INFO:teuthology.orchestra.run.smithi122.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-16T20:41:26.753 INFO:teuthology.orchestra.run.smithi122.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-16T20:41:26.754 INFO:teuthology.orchestra.run.smithi122.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-16T20:41:26.754 INFO:teuthology.orchestra.run.smithi122.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-16T20:41:26.755 INFO:teuthology.orchestra.run.smithi122.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-16T20:41:26.755 INFO:teuthology.orchestra.run.smithi122.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-16T20:41:26.755 INFO:teuthology.orchestra.run.smithi122.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-16T20:41:26.776 INFO:teuthology.orchestra.run.smithi122.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-16T20:41:26.777 INFO:teuthology.orchestra.run.smithi122.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-16T20:41:26.788 INFO:teuthology.orchestra.run.smithi122.stdout:Get:39 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1681-gaba3c1d2-1jammy [3,546 kB] 2024-03-16T20:41:26.800 INFO:teuthology.orchestra.run.smithi122.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-16T20:41:26.806 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-16T20:41:26.811 INFO:teuthology.orchestra.run.smithi122.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-16T20:41:26.811 INFO:teuthology.orchestra.run.smithi122.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-16T20:41:26.812 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-16T20:41:26.812 INFO:teuthology.orchestra.run.smithi122.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-16T20:41:26.813 INFO:teuthology.orchestra.run.smithi122.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-16T20:41:26.816 INFO:teuthology.orchestra.run.smithi122.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-16T20:41:26.820 INFO:teuthology.orchestra.run.smithi122.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-16T20:41:26.821 INFO:teuthology.orchestra.run.smithi122.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-16T20:41:26.829 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-16T20:41:26.832 INFO:teuthology.orchestra.run.smithi122.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-16T20:41:26.834 INFO:teuthology.orchestra.run.smithi122.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-16T20:41:26.834 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-16T20:41:26.835 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-16T20:41:26.847 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-16T20:41:26.850 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-16T20:41:26.852 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-16T20:41:26.854 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-16T20:41:26.854 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-16T20:41:26.856 INFO:teuthology.orchestra.run.smithi122.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-16T20:41:26.857 INFO:teuthology.orchestra.run.smithi122.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-16T20:41:26.858 INFO:teuthology.orchestra.run.smithi122.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-16T20:41:26.863 INFO:teuthology.orchestra.run.smithi122.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-16T20:41:26.875 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-16T20:41:26.880 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-16T20:41:26.884 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-16T20:41:26.918 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-16T20:41:26.918 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-16T20:41:26.920 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-16T20:41:26.921 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-16T20:41:26.922 INFO:teuthology.orchestra.run.smithi195.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-16T20:41:26.922 INFO:teuthology.orchestra.run.smithi195.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-16T20:41:26.923 INFO:teuthology.orchestra.run.smithi195.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-16T20:41:26.939 INFO:teuthology.orchestra.run.smithi122.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-16T20:41:26.941 INFO:teuthology.orchestra.run.smithi122.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-16T20:41:26.942 INFO:teuthology.orchestra.run.smithi122.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-16T20:41:26.942 INFO:teuthology.orchestra.run.smithi122.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-16T20:41:26.944 INFO:teuthology.orchestra.run.smithi122.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-16T20:41:26.961 INFO:teuthology.orchestra.run.smithi195.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-16T20:41:26.962 INFO:teuthology.orchestra.run.smithi195.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-16T20:41:26.967 INFO:teuthology.orchestra.run.smithi195.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-16T20:41:26.978 INFO:teuthology.orchestra.run.smithi122.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-16T20:41:26.980 INFO:teuthology.orchestra.run.smithi195.stdout:Get:31 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1681-gaba3c1d2-1jammy [3,546 kB] 2024-03-16T20:41:26.980 INFO:teuthology.orchestra.run.smithi122.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-16T20:41:26.981 INFO:teuthology.orchestra.run.smithi122.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-16T20:41:27.006 INFO:teuthology.orchestra.run.smithi122.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-16T20:41:27.006 INFO:teuthology.orchestra.run.smithi122.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-16T20:41:27.007 INFO:teuthology.orchestra.run.smithi195.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-16T20:41:27.007 INFO:teuthology.orchestra.run.smithi195.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-16T20:41:27.008 INFO:teuthology.orchestra.run.smithi195.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-16T20:41:27.010 INFO:teuthology.orchestra.run.smithi195.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-16T20:41:27.010 INFO:teuthology.orchestra.run.smithi195.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-16T20:41:27.011 INFO:teuthology.orchestra.run.smithi195.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-16T20:41:27.013 INFO:teuthology.orchestra.run.smithi195.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-16T20:41:27.021 INFO:teuthology.orchestra.run.smithi122.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-16T20:41:27.022 INFO:teuthology.orchestra.run.smithi122.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-16T20:41:27.022 INFO:teuthology.orchestra.run.smithi122.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-16T20:41:27.023 INFO:teuthology.orchestra.run.smithi122.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-16T20:41:27.023 INFO:teuthology.orchestra.run.smithi122.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-16T20:41:27.024 INFO:teuthology.orchestra.run.smithi122.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-16T20:41:27.028 INFO:teuthology.orchestra.run.smithi122.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-16T20:41:27.035 INFO:teuthology.orchestra.run.smithi195.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-16T20:41:27.036 INFO:teuthology.orchestra.run.smithi195.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-16T20:41:27.046 INFO:teuthology.orchestra.run.smithi122.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-16T20:41:27.047 INFO:teuthology.orchestra.run.smithi195.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-16T20:41:27.048 INFO:teuthology.orchestra.run.smithi122.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-16T20:41:27.049 INFO:teuthology.orchestra.run.smithi122.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-16T20:41:27.060 INFO:teuthology.orchestra.run.smithi195.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-16T20:41:27.061 INFO:teuthology.orchestra.run.smithi195.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-16T20:41:27.062 INFO:teuthology.orchestra.run.smithi195.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-16T20:41:27.065 INFO:teuthology.orchestra.run.smithi195.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-16T20:41:27.071 INFO:teuthology.orchestra.run.smithi122.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-16T20:41:27.075 INFO:teuthology.orchestra.run.smithi122.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-16T20:41:27.075 INFO:teuthology.orchestra.run.smithi122.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-16T20:41:27.077 INFO:teuthology.orchestra.run.smithi122.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-16T20:41:27.077 INFO:teuthology.orchestra.run.smithi122.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-16T20:41:27.083 INFO:teuthology.orchestra.run.smithi122.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-16T20:41:27.084 INFO:teuthology.orchestra.run.smithi122.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-16T20:41:27.099 INFO:teuthology.orchestra.run.smithi122.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-16T20:41:27.100 INFO:teuthology.orchestra.run.smithi122.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-16T20:41:27.101 INFO:teuthology.orchestra.run.smithi122.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-16T20:41:27.108 INFO:teuthology.orchestra.run.smithi195.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-16T20:41:27.109 INFO:teuthology.orchestra.run.smithi195.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-16T20:41:27.112 INFO:teuthology.orchestra.run.smithi195.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-16T20:41:27.113 INFO:teuthology.orchestra.run.smithi195.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-16T20:41:27.118 INFO:teuthology.orchestra.run.smithi195.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-16T20:41:27.119 INFO:teuthology.orchestra.run.smithi195.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-16T20:41:27.120 INFO:teuthology.orchestra.run.smithi195.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-16T20:41:27.122 INFO:teuthology.orchestra.run.smithi122.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-16T20:41:27.124 INFO:teuthology.orchestra.run.smithi122.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-16T20:41:27.125 INFO:teuthology.orchestra.run.smithi195.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-16T20:41:27.125 INFO:teuthology.orchestra.run.smithi122.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-16T20:41:27.136 INFO:teuthology.orchestra.run.smithi122.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-16T20:41:27.136 INFO:teuthology.orchestra.run.smithi122.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-16T20:41:27.136 INFO:teuthology.orchestra.run.smithi122.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-16T20:41:27.140 INFO:teuthology.orchestra.run.smithi122.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-16T20:41:27.175 INFO:teuthology.orchestra.run.smithi122.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-16T20:41:27.290 INFO:teuthology.orchestra.run.smithi195.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-16T20:41:27.290 INFO:teuthology.orchestra.run.smithi195.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-16T20:41:27.292 INFO:teuthology.orchestra.run.smithi195.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-16T20:41:27.292 INFO:teuthology.orchestra.run.smithi195.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-16T20:41:27.293 INFO:teuthology.orchestra.run.smithi195.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-16T20:41:27.385 INFO:teuthology.orchestra.run.smithi195.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-16T20:41:27.390 INFO:teuthology.orchestra.run.smithi195.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-16T20:41:27.390 INFO:teuthology.orchestra.run.smithi195.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-16T20:41:27.423 INFO:teuthology.orchestra.run.smithi195.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-16T20:41:27.423 INFO:teuthology.orchestra.run.smithi195.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-16T20:41:27.424 INFO:teuthology.orchestra.run.smithi195.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-16T20:41:27.424 INFO:teuthology.orchestra.run.smithi195.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-16T20:41:27.425 INFO:teuthology.orchestra.run.smithi195.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-16T20:41:27.426 INFO:teuthology.orchestra.run.smithi195.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-16T20:41:27.426 INFO:teuthology.orchestra.run.smithi195.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-16T20:41:27.434 INFO:teuthology.orchestra.run.smithi195.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-16T20:41:27.443 INFO:teuthology.orchestra.run.smithi195.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-16T20:41:27.456 INFO:teuthology.orchestra.run.smithi195.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-16T20:41:27.458 INFO:teuthology.orchestra.run.smithi195.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-16T20:41:27.460 INFO:teuthology.orchestra.run.smithi195.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-16T20:41:27.469 INFO:teuthology.orchestra.run.smithi195.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-16T20:41:27.476 INFO:teuthology.orchestra.run.smithi195.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-03-16T20:41:27.476 INFO:teuthology.orchestra.run.smithi195.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-16T20:41:27.477 INFO:teuthology.orchestra.run.smithi195.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-16T20:41:27.478 INFO:teuthology.orchestra.run.smithi195.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-16T20:41:27.488 INFO:teuthology.orchestra.run.smithi195.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-16T20:41:27.489 INFO:teuthology.orchestra.run.smithi195.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-16T20:41:27.508 INFO:teuthology.orchestra.run.smithi195.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-16T20:41:27.508 INFO:teuthology.orchestra.run.smithi195.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-16T20:41:27.509 INFO:teuthology.orchestra.run.smithi195.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-16T20:41:27.509 INFO:teuthology.orchestra.run.smithi195.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-16T20:41:27.513 INFO:teuthology.orchestra.run.smithi195.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-16T20:41:27.515 INFO:teuthology.orchestra.run.smithi195.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-16T20:41:27.530 INFO:teuthology.orchestra.run.smithi122.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1681-gaba3c1d2-1jammy [3,211 kB] 2024-03-16T20:41:27.534 INFO:teuthology.orchestra.run.smithi195.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-16T20:41:27.535 INFO:teuthology.orchestra.run.smithi195.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-16T20:41:27.536 INFO:teuthology.orchestra.run.smithi195.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-16T20:41:27.540 INFO:teuthology.orchestra.run.smithi195.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-16T20:41:27.557 INFO:teuthology.orchestra.run.smithi195.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-16T20:41:27.950 INFO:teuthology.orchestra.run.smithi122.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1681-gaba3c1d2-1jammy [709 kB] 2024-03-16T20:41:28.055 INFO:teuthology.orchestra.run.smithi122.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1681-gaba3c1d2-1jammy [356 kB] 2024-03-16T20:41:28.104 INFO:teuthology.orchestra.run.smithi122.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1681-gaba3c1d2-1jammy [32.8 kB] 2024-03-16T20:41:28.105 INFO:teuthology.orchestra.run.smithi122.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1681-gaba3c1d2-1jammy [180 kB] 2024-03-16T20:41:28.114 INFO:teuthology.orchestra.run.smithi122.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1681-gaba3c1d2-1jammy [64.8 kB] 2024-03-16T20:41:28.116 INFO:teuthology.orchestra.run.smithi122.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1681-gaba3c1d2-1jammy [330 kB] 2024-03-16T20:41:28.170 INFO:teuthology.orchestra.run.smithi122.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1681-gaba3c1d2-1jammy [6,570 kB] 2024-03-16T20:41:29.080 INFO:teuthology.orchestra.run.smithi122.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1681-gaba3c1d2-1jammy [112 kB] 2024-03-16T20:41:29.097 INFO:teuthology.orchestra.run.smithi122.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1681-gaba3c1d2-1jammy [465 kB] 2024-03-16T20:41:29.149 INFO:teuthology.orchestra.run.smithi122.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1681-gaba3c1d2-1jammy [24.9 MB] 2024-03-16T20:41:29.277 INFO:teuthology.orchestra.run.smithi195.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1681-gaba3c1d2-1jammy [3,211 kB] 2024-03-16T20:41:30.131 INFO:teuthology.orchestra.run.smithi195.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1681-gaba3c1d2-1jammy [709 kB] 2024-03-16T20:41:30.298 INFO:teuthology.orchestra.run.smithi195.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1681-gaba3c1d2-1jammy [356 kB] 2024-03-16T20:41:30.336 INFO:teuthology.orchestra.run.smithi195.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1681-gaba3c1d2-1jammy [32.8 kB] 2024-03-16T20:41:30.339 INFO:teuthology.orchestra.run.smithi195.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1681-gaba3c1d2-1jammy [180 kB] 2024-03-16T20:41:30.407 INFO:teuthology.orchestra.run.smithi195.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1681-gaba3c1d2-1jammy [64.8 kB] 2024-03-16T20:41:30.411 INFO:teuthology.orchestra.run.smithi195.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1681-gaba3c1d2-1jammy [330 kB] 2024-03-16T20:41:30.489 INFO:teuthology.orchestra.run.smithi195.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1681-gaba3c1d2-1jammy [6,570 kB] 2024-03-16T20:41:31.785 INFO:teuthology.orchestra.run.smithi195.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1681-gaba3c1d2-1jammy [112 kB] 2024-03-16T20:41:31.836 INFO:teuthology.orchestra.run.smithi195.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1681-gaba3c1d2-1jammy [465 kB] 2024-03-16T20:41:31.925 INFO:teuthology.orchestra.run.smithi195.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1681-gaba3c1d2-1jammy [24.9 MB] 2024-03-16T20:41:32.463 INFO:teuthology.orchestra.run.smithi122.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1681-gaba3c1d2-1jammy [5,074 kB] 2024-03-16T20:41:33.025 INFO:teuthology.orchestra.run.smithi122.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1681-gaba3c1d2-1jammy [244 kB] 2024-03-16T20:41:33.061 INFO:teuthology.orchestra.run.smithi122.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1681-gaba3c1d2-1jammy [124 kB] 2024-03-16T20:41:33.066 INFO:teuthology.orchestra.run.smithi122.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1681-gaba3c1d2-1jammy [1,522 kB] 2024-03-16T20:41:33.278 INFO:teuthology.orchestra.run.smithi122.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1681-gaba3c1d2-1jammy [6,243 kB] 2024-03-16T20:41:34.089 INFO:teuthology.orchestra.run.smithi122.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1681-gaba3c1d2-1jammy [22.7 MB] 2024-03-16T20:41:35.945 INFO:teuthology.orchestra.run.smithi195.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1681-gaba3c1d2-1jammy [5,074 kB] 2024-03-16T20:41:36.315 INFO:teuthology.orchestra.run.smithi122.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1681-gaba3c1d2-1jammy [14.0 kB] 2024-03-16T20:41:36.315 INFO:teuthology.orchestra.run.smithi122.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1681-gaba3c1d2-1jammy [867 kB] 2024-03-16T20:41:36.332 INFO:teuthology.orchestra.run.smithi122.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1681-gaba3c1d2-1jammy [2,378 kB] 2024-03-16T20:41:36.385 INFO:teuthology.orchestra.run.smithi195.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1681-gaba3c1d2-1jammy [244 kB] 2024-03-16T20:41:36.417 INFO:teuthology.orchestra.run.smithi195.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1681-gaba3c1d2-1jammy [124 kB] 2024-03-16T20:41:36.434 INFO:teuthology.orchestra.run.smithi195.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1681-gaba3c1d2-1jammy [1,522 kB] 2024-03-16T20:41:36.469 INFO:teuthology.orchestra.run.smithi122.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1681-gaba3c1d2-1jammy [776 kB] 2024-03-16T20:41:36.517 INFO:teuthology.orchestra.run.smithi122.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1681-gaba3c1d2-1jammy [144 kB] 2024-03-16T20:41:36.530 INFO:teuthology.orchestra.run.smithi122.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1681-gaba3c1d2-1jammy [1,921 kB] 2024-03-16T20:41:36.610 INFO:teuthology.orchestra.run.smithi195.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1681-gaba3c1d2-1jammy [6,243 kB] 2024-03-16T20:41:36.670 INFO:teuthology.orchestra.run.smithi122.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1681-gaba3c1d2-1jammy [8,625 kB] 2024-03-16T20:41:37.290 INFO:teuthology.orchestra.run.smithi195.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1681-gaba3c1d2-1jammy [22.7 MB] 2024-03-16T20:41:37.293 INFO:teuthology.orchestra.run.smithi122.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1681-gaba3c1d2-1jammy [14.2 kB] 2024-03-16T20:41:37.294 INFO:teuthology.orchestra.run.smithi122.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1681-gaba3c1d2-1jammy [49.9 MB] 2024-03-16T20:41:39.998 INFO:teuthology.orchestra.run.smithi195.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1681-gaba3c1d2-1jammy [14.0 kB] 2024-03-16T20:41:39.998 INFO:teuthology.orchestra.run.smithi195.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1681-gaba3c1d2-1jammy [867 kB] 2024-03-16T20:41:40.015 INFO:teuthology.orchestra.run.smithi195.stdout:Get:111 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1681-gaba3c1d2-1jammy [2,378 kB] 2024-03-16T20:41:40.288 INFO:teuthology.orchestra.run.smithi195.stdout:Get:112 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1681-gaba3c1d2-1jammy [776 kB] 2024-03-16T20:41:40.382 INFO:teuthology.orchestra.run.smithi195.stdout:Get:113 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1681-gaba3c1d2-1jammy [144 kB] 2024-03-16T20:41:40.398 INFO:teuthology.orchestra.run.smithi195.stdout:Get:114 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1681-gaba3c1d2-1jammy [1,921 kB] 2024-03-16T20:41:40.631 INFO:teuthology.orchestra.run.smithi195.stdout:Get:115 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1681-gaba3c1d2-1jammy [8,625 kB] 2024-03-16T20:41:41.247 INFO:teuthology.orchestra.run.smithi122.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1681-gaba3c1d2-1jammy [121 kB] 2024-03-16T20:41:41.250 INFO:teuthology.orchestra.run.smithi122.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1681-gaba3c1d2-1jammy [226 kB] 2024-03-16T20:41:41.255 INFO:teuthology.orchestra.run.smithi122.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1681-gaba3c1d2-1jammy [33.9 kB] 2024-03-16T20:41:41.255 INFO:teuthology.orchestra.run.smithi122.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1681-gaba3c1d2-1jammy [23.4 kB] 2024-03-16T20:41:41.256 INFO:teuthology.orchestra.run.smithi122.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1681-gaba3c1d2-1jammy [38.5 kB] 2024-03-16T20:41:41.257 INFO:teuthology.orchestra.run.smithi122.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1681-gaba3c1d2-1jammy [13.0 MB] 2024-03-16T20:41:41.700 INFO:teuthology.orchestra.run.smithi195.stdout:Get:116 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1681-gaba3c1d2-1jammy [14.2 kB] 2024-03-16T20:41:41.700 INFO:teuthology.orchestra.run.smithi195.stdout:Get:117 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1681-gaba3c1d2-1jammy [49.9 MB] 2024-03-16T20:41:42.097 INFO:teuthology.orchestra.run.smithi122.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1681-gaba3c1d2-1jammy [91.7 kB] 2024-03-16T20:41:42.704 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 177 MB in 16s (11.3 MB/s) 2024-03-16T20:41:42.842 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-16T20:41:42.937 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-16T20:41:42.942 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-16T20:41:42.983 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-16T20:41:43.261 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-16T20:41:43.287 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-16T20:41:43.319 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-16T20:41:43.631 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cmd2. 2024-03-16T20:41:43.657 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-16T20:41:43.697 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-16T20:41:44.042 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-16T20:41:44.066 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-16T20:41:44.151 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-16T20:41:44.522 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-16T20:41:44.548 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-16T20:41:44.590 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-16T20:41:44.892 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-16T20:41:44.918 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-16T20:41:44.950 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-16T20:41:45.379 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-16T20:41:45.402 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-16T20:41:45.454 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T20:41:45.591 INFO:teuthology.orchestra.run.smithi195.stdout:Get:118 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1681-gaba3c1d2-1jammy [121 kB] 2024-03-16T20:41:45.594 INFO:teuthology.orchestra.run.smithi195.stdout:Get:119 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-1681-gaba3c1d2-1jammy [226 kB] 2024-03-16T20:41:45.598 INFO:teuthology.orchestra.run.smithi195.stdout:Get:120 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-1681-gaba3c1d2-1jammy [33.9 kB] 2024-03-16T20:41:45.599 INFO:teuthology.orchestra.run.smithi195.stdout:Get:121 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-1681-gaba3c1d2-1jammy [23.4 kB] 2024-03-16T20:41:45.600 INFO:teuthology.orchestra.run.smithi195.stdout:Get:122 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1681-gaba3c1d2-1jammy [38.5 kB] 2024-03-16T20:41:45.601 INFO:teuthology.orchestra.run.smithi195.stdout:Get:123 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1681-gaba3c1d2-1jammy [13.0 MB] 2024-03-16T20:41:45.849 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-16T20:41:45.876 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-16T20:41:45.917 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T20:41:46.088 INFO:teuthology.orchestra.run.smithi195.stdout:Get:124 https://2.chacra.ceph.com/r/ceph/wip-lusov-qdb-squid/aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1681-gaba3c1d2-1jammy [91.7 kB] 2024-03-16T20:41:46.269 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-16T20:41:46.296 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-16T20:41:46.336 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T20:41:46.673 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-16T20:41:46.692 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 177 MB in 20s (9,045 kB/s) 2024-03-16T20:41:46.699 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-16T20:41:46.739 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-16T20:41:46.802 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyperclip. 2024-03-16T20:41:46.895 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165997 files and directories currently installed.) 2024-03-16T20:41:46.900 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-03-16T20:41:46.935 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-03-16T20:41:47.051 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librados2. 2024-03-16T20:41:47.077 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../010-librados2_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:47.118 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librados2 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:47.196 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-16T20:41:47.223 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-16T20:41:47.263 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-16T20:41:47.582 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cmd2. 2024-03-16T20:41:47.609 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-03-16T20:41:47.649 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-03-16T20:41:47.771 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librbd1. 2024-03-16T20:41:47.797 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../011-librbd1_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:47.837 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librbd1 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:48.002 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-16T20:41:48.029 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-16T20:41:48.111 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-16T20:41:48.258 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libcephfs2. 2024-03-16T20:41:48.284 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:48.316 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libcephfs2 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:48.447 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-16T20:41:48.474 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-16T20:41:48.514 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-16T20:41:48.577 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rados. 2024-03-16T20:41:48.603 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:48.644 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rados (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:48.783 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-16T20:41:48.810 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-16T20:41:48.842 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-16T20:41:48.963 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-16T20:41:48.990 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:41:49.030 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-ceph-argparse (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:49.291 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cephfs. 2024-03-16T20:41:49.297 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-16T20:41:49.317 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:49.322 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-16T20:41:49.358 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cephfs (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:49.427 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T20:41:49.603 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-16T20:41:49.629 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:41:49.661 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-ceph-common (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:49.748 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-16T20:41:49.775 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-16T20:41:49.815 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T20:41:49.964 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-prettytable. 2024-03-16T20:41:49.990 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-16T20:41:50.031 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-16T20:41:50.143 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-16T20:41:50.170 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-16T20:41:50.210 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T20:41:50.292 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rbd. 2024-03-16T20:41:50.308 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:50.342 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rbd (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:50.563 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-16T20:41:50.589 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-16T20:41:50.629 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-16T20:41:50.670 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-16T20:41:50.697 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-16T20:41:50.737 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-16T20:41:50.932 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librados2. 2024-03-16T20:41:50.947 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../010-librados2_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:50.982 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librados2 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:51.015 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-16T20:41:51.041 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-16T20:41:51.081 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-16T20:41:51.393 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-16T20:41:51.409 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-16T20:41:51.410 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librbd1. 2024-03-16T20:41:51.436 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../011-librbd1_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:51.443 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-16T20:41:51.476 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librbd1 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:51.765 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua5.1. 2024-03-16T20:41:51.790 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-16T20:41:51.830 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-16T20:41:52.063 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcephfs2. 2024-03-16T20:41:52.089 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:52.129 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcephfs2 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:52.150 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua-any. 2024-03-16T20:41:52.165 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-16T20:41:52.209 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-16T20:41:52.415 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rados. 2024-03-16T20:41:52.442 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../013-python3-rados_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:52.453 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package zip. 2024-03-16T20:41:52.468 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-16T20:41:52.481 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rados (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:52.520 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking zip (3.0-12build2) ... 2024-03-16T20:41:52.784 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-16T20:41:52.811 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:41:52.851 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-ceph-argparse (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:52.856 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package unzip. 2024-03-16T20:41:52.883 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-16T20:41:52.931 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-16T20:41:53.104 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cephfs. 2024-03-16T20:41:53.130 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:53.170 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cephfs (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:53.394 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package luarocks. 2024-03-16T20:41:53.421 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-16T20:41:53.461 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-16T20:41:53.489 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-16T20:41:53.516 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:41:53.556 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-ceph-common (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:53.845 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librgw2. 2024-03-16T20:41:53.871 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../027-librgw2_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:53.909 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-prettytable. 2024-03-16T20:41:53.911 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librgw2 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:53.935 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-03-16T20:41:53.976 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-16T20:41:54.220 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rbd. 2024-03-16T20:41:54.235 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:54.270 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rbd (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:54.472 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rgw. 2024-03-16T20:41:54.488 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:54.539 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rgw (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:54.598 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-16T20:41:54.624 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-16T20:41:54.664 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-16T20:41:54.942 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-16T20:41:54.969 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-16T20:41:54.975 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-16T20:41:54.990 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-16T20:41:55.009 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-16T20:41:55.033 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-16T20:41:55.321 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libradosstriper1. 2024-03-16T20:41:55.337 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-16T20:41:55.347 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:55.352 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-16T20:41:55.395 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-16T20:41:55.406 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libradosstriper1 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:55.698 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package lua5.1. 2024-03-16T20:41:55.713 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-16T20:41:55.748 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-16T20:41:55.924 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-common. 2024-03-16T20:41:55.950 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:55.999 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-common (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:56.068 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package lua-any. 2024-03-16T20:41:56.094 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-03-16T20:41:56.135 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-16T20:41:56.362 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package zip. 2024-03-16T20:41:56.388 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-03-16T20:41:56.428 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking zip (3.0-12build2) ... 2024-03-16T20:41:56.715 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package unzip. 2024-03-16T20:41:56.741 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-16T20:41:56.781 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-16T20:41:57.247 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-base. 2024-03-16T20:41:57.269 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package luarocks. 2024-03-16T20:41:57.273 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:57.295 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-16T20:41:57.315 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-base (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:57.327 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-16T20:41:57.718 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package librgw2. 2024-03-16T20:41:57.745 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../027-librgw2_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:57.785 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking librgw2 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:57.827 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-16T20:41:57.853 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-16T20:41:57.893 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-16T20:41:58.354 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rgw. 2024-03-16T20:41:58.380 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cheroot. 2024-03-16T20:41:58.380 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:58.407 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-16T20:41:58.420 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rgw (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:58.446 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-16T20:41:58.732 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-16T20:41:58.758 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-16T20:41:58.798 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-16T20:41:59.043 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libradosstriper1. 2024-03-16T20:41:59.058 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:59.101 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libradosstriper1 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:59.509 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-16T20:41:59.529 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-common. 2024-03-16T20:41:59.537 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-16T20:41:59.556 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../031-ceph-common_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:41:59.575 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-16T20:41:59.596 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-common (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:41:59.854 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-16T20:41:59.881 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-16T20:41:59.920 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-16T20:42:00.198 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-16T20:42:00.225 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-16T20:42:00.265 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-16T20:42:00.570 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-tempora. 2024-03-16T20:42:00.595 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-16T20:42:00.635 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-16T20:42:00.913 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-portend. 2024-03-16T20:42:00.940 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-16T20:42:00.980 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-16T20:42:01.258 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-16T20:42:01.284 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-16T20:42:01.333 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-16T20:42:01.610 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-base. 2024-03-16T20:42:01.628 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-16T20:42:01.636 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../032-ceph-base_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:01.654 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-16T20:42:01.677 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-base (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:01.695 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-16T20:42:02.023 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-natsort. 2024-03-16T20:42:02.050 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-16T20:42:02.081 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-16T20:42:02.402 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-logutils. 2024-03-16T20:42:02.429 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-16T20:42:02.439 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-16T20:42:02.466 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-16T20:42:02.468 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-16T20:42:02.506 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-16T20:42:02.772 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-16T20:42:02.783 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cheroot. 2024-03-16T20:42:02.799 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-16T20:42:02.810 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-16T20:42:02.838 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-16T20:42:02.850 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-16T20:42:03.124 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-mako. 2024-03-16T20:42:03.136 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-16T20:42:03.151 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-16T20:42:03.163 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-16T20:42:03.191 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-16T20:42:03.195 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-16T20:42:03.481 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-16T20:42:03.487 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-16T20:42:03.507 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-16T20:42:03.513 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-16T20:42:03.545 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-16T20:42:03.547 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-16T20:42:03.848 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-16T20:42:03.850 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-16T20:42:03.875 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-16T20:42:03.877 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-16T20:42:03.915 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-16T20:42:03.917 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-16T20:42:04.220 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-tempora. 2024-03-16T20:42:04.226 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-webob. 2024-03-16T20:42:04.236 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-03-16T20:42:04.253 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-16T20:42:04.278 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-16T20:42:04.293 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-16T20:42:04.564 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-portend. 2024-03-16T20:42:04.591 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-03-16T20:42:04.631 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-16T20:42:04.646 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-waitress. 2024-03-16T20:42:04.673 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-16T20:42:04.715 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-16T20:42:04.917 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-16T20:42:04.944 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-16T20:42:04.975 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-16T20:42:04.999 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-tempita. 2024-03-16T20:42:05.026 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-16T20:42:05.057 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-16T20:42:05.237 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-16T20:42:05.263 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-16T20:42:05.303 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-16T20:42:05.344 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-paste. 2024-03-16T20:42:05.371 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-16T20:42:05.411 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-16T20:42:05.640 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-natsort. 2024-03-16T20:42:05.655 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-03-16T20:42:05.698 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-16T20:42:05.723 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-16T20:42:05.749 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-16T20:42:05.789 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-16T20:42:06.010 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-logutils. 2024-03-16T20:42:06.036 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-03-16T20:42:06.076 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-16T20:42:06.076 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-16T20:42:06.103 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-16T20:42:06.143 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-16T20:42:06.362 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-16T20:42:06.389 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-16T20:42:06.420 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-16T20:42:06.446 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-webtest. 2024-03-16T20:42:06.473 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-16T20:42:06.513 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-16T20:42:06.665 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-mako. 2024-03-16T20:42:06.692 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-16T20:42:06.733 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-16T20:42:06.824 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pecan. 2024-03-16T20:42:06.851 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-16T20:42:06.891 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-16T20:42:07.043 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-16T20:42:07.070 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-16T20:42:07.118 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-16T20:42:07.263 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-certifi. 2024-03-16T20:42:07.290 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-16T20:42:07.329 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-16T20:42:07.421 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-16T20:42:07.447 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-16T20:42:07.487 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-16T20:42:07.582 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-idna. 2024-03-16T20:42:07.609 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-16T20:42:07.649 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-16T20:42:07.758 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-webob. 2024-03-16T20:42:07.772 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-16T20:42:07.806 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-16T20:42:07.988 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-urllib3. 2024-03-16T20:42:08.015 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-16T20:42:08.054 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-16T20:42:08.109 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-waitress. 2024-03-16T20:42:08.125 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-16T20:42:08.188 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-16T20:42:08.366 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-requests. 2024-03-16T20:42:08.393 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-16T20:42:08.435 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-16T20:42:08.446 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-tempita. 2024-03-16T20:42:08.472 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-16T20:42:08.512 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-16T20:42:08.815 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-paste. 2024-03-16T20:42:08.841 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-16T20:42:08.882 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-16T20:42:08.894 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-16T20:42:08.921 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-16T20:42:08.969 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-16T20:42:09.210 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-16T20:42:09.225 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-16T20:42:09.256 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-16T20:42:09.268 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-16T20:42:09.283 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:42:09.322 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:09.538 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-16T20:42:09.565 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-16T20:42:09.596 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-16T20:42:09.695 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-16T20:42:09.722 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:09.761 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:09.866 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-webtest. 2024-03-16T20:42:09.892 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-03-16T20:42:09.932 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-16T20:42:10.073 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-16T20:42:10.100 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-16T20:42:10.139 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-16T20:42:10.227 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pecan. 2024-03-16T20:42:10.253 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-16T20:42:10.295 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-16T20:42:10.451 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-openssl. 2024-03-16T20:42:10.478 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-16T20:42:10.517 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-16T20:42:10.658 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-certifi. 2024-03-16T20:42:10.685 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-03-16T20:42:10.724 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-16T20:42:10.821 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr. 2024-03-16T20:42:10.849 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:10.887 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:11.028 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-idna. 2024-03-16T20:42:11.054 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-03-16T20:42:11.086 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-16T20:42:11.283 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mon. 2024-03-16T20:42:11.310 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:11.349 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mon (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:11.381 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-urllib3. 2024-03-16T20:42:11.407 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-16T20:42:11.447 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-16T20:42:11.784 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-requests. 2024-03-16T20:42:11.811 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-16T20:42:11.850 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-16T20:42:11.978 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-osd. 2024-03-16T20:42:12.006 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:12.045 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-osd (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:12.161 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-16T20:42:12.188 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-16T20:42:12.228 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-16T20:42:12.685 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-16T20:42:12.712 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:42:12.748 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:13.082 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph. 2024-03-16T20:42:13.097 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../068-ceph_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:13.119 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-16T20:42:13.140 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:13.146 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:13.185 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:13.460 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-fuse. 2024-03-16T20:42:13.480 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-16T20:42:13.487 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:13.507 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-16T20:42:13.526 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-fuse (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:13.547 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-16T20:42:13.850 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-openssl. 2024-03-16T20:42:13.877 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-03-16T20:42:13.916 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-16T20:42:13.980 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mds. 2024-03-16T20:42:14.007 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:14.046 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mds (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:14.269 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr. 2024-03-16T20:42:14.296 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:14.335 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:14.399 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package cephadm. 2024-03-16T20:42:14.426 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../071-cephadm_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:14.466 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking cephadm (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:14.697 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mon. 2024-03-16T20:42:14.724 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:14.755 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mon (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:14.785 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-16T20:42:14.813 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-16T20:42:14.885 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-16T20:42:15.205 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-16T20:42:15.233 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-16T20:42:15.272 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-16T20:42:15.343 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-osd. 2024-03-16T20:42:15.370 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:15.409 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-osd (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:16.286 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-babel. 2024-03-16T20:42:16.314 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-16T20:42:16.344 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-16T20:42:16.470 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph. 2024-03-16T20:42:16.485 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../068-ceph_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:16.528 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:16.614 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jinja2. 2024-03-16T20:42:16.641 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-16T20:42:16.681 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-16T20:42:16.814 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-fuse. 2024-03-16T20:42:16.841 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:16.881 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-fuse (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:16.968 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-16T20:42:16.995 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:42:17.034 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:17.226 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mds. 2024-03-16T20:42:17.252 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:17.292 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mds (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:17.346 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-16T20:42:17.374 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-16T20:42:17.413 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-16T20:42:17.687 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package cephadm. 2024-03-16T20:42:17.714 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../071-cephadm_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:17.716 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-routes. 2024-03-16T20:42:17.743 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-16T20:42:17.753 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking cephadm (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:17.782 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-16T20:42:18.111 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-16T20:42:18.138 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:42:18.177 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:18.248 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-16T20:42:18.275 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-16T20:42:18.314 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-16T20:42:18.615 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-16T20:42:18.642 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-16T20:42:18.643 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-16T20:42:18.669 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-16T20:42:18.681 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-16T20:42:18.709 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-16T20:42:19.143 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-joblib. 2024-03-16T20:42:19.159 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-16T20:42:19.193 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-16T20:42:19.581 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-16T20:42:19.608 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-16T20:42:19.733 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-16T20:42:19.764 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-babel. 2024-03-16T20:42:19.791 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-16T20:42:19.830 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-16T20:42:20.020 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-sklearn. 2024-03-16T20:42:20.047 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-16T20:42:20.086 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-16T20:42:20.172 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-jinja2. 2024-03-16T20:42:20.172 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-16T20:42:20.199 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-16T20:42:20.653 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-16T20:42:20.680 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:42:20.719 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:20.797 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-16T20:42:20.825 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:42:20.863 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:21.048 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-16T20:42:21.075 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-03-16T20:42:21.114 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-16T20:42:21.417 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-routes. 2024-03-16T20:42:21.444 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-16T20:42:21.508 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-16T20:42:21.853 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-16T20:42:21.881 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:42:21.900 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cachetools. 2024-03-16T20:42:21.919 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:21.927 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-16T20:42:21.966 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-16T20:42:22.261 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-16T20:42:22.289 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-16T20:42:22.290 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-16T20:42:22.305 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-16T20:42:22.328 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-16T20:42:22.339 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-16T20:42:22.640 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-16T20:42:22.667 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-16T20:42:22.698 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-16T20:42:22.868 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-joblib. 2024-03-16T20:42:22.895 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-16T20:42:22.934 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-16T20:42:22.960 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rsa. 2024-03-16T20:42:22.987 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-16T20:42:23.018 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-16T20:42:23.347 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-google-auth. 2024-03-16T20:42:23.375 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-16T20:42:23.414 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-16T20:42:23.463 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-16T20:42:23.490 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-16T20:42:23.615 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-16T20:42:23.717 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-16T20:42:23.745 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-16T20:42:23.784 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-16T20:42:23.918 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-sklearn. 2024-03-16T20:42:23.945 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-16T20:42:23.984 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-16T20:42:24.288 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-websocket. 2024-03-16T20:42:24.315 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-16T20:42:24.346 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-16T20:42:24.633 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-16T20:42:24.661 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-16T20:42:24.677 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-16T20:42:24.699 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-16T20:42:24.705 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:42:24.743 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:25.374 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-16T20:42:25.388 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:42:25.431 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:25.730 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-cachetools. 2024-03-16T20:42:25.750 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-16T20:42:25.758 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-03-16T20:42:25.777 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-16T20:42:25.797 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-16T20:42:25.817 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-16T20:42:26.083 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-16T20:42:26.110 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-16T20:42:26.112 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-16T20:42:26.139 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-16T20:42:26.149 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-16T20:42:26.179 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-16T20:42:26.399 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package jq. 2024-03-16T20:42:26.416 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-16T20:42:26.448 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-16T20:42:26.486 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-16T20:42:26.513 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-16T20:42:26.552 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-16T20:42:26.660 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package socat. 2024-03-16T20:42:26.676 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-16T20:42:26.709 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-16T20:42:26.839 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-rsa. 2024-03-16T20:42:26.866 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-03-16T20:42:26.905 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-16T20:42:27.022 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package xmlstarlet. 2024-03-16T20:42:27.049 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-16T20:42:27.088 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-16T20:42:27.201 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-google-auth. 2024-03-16T20:42:27.228 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-03-16T20:42:27.267 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-16T20:42:27.443 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-test. 2024-03-16T20:42:27.459 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:27.501 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-test (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:27.570 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-16T20:42:27.598 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-16T20:42:27.636 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-16T20:42:27.948 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-websocket. 2024-03-16T20:42:27.976 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-03-16T20:42:28.014 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-16T20:42:28.360 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-16T20:42:28.387 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-16T20:42:28.426 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-16T20:42:29.115 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-16T20:42:29.131 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:42:29.173 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:29.501 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-16T20:42:29.528 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-16T20:42:29.567 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-16T20:42:29.798 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-volume. 2024-03-16T20:42:29.826 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:42:29.854 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-16T20:42:29.864 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-volume (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:29.882 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-16T20:42:29.920 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-16T20:42:30.182 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package jq. 2024-03-16T20:42:30.210 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-16T20:42:30.236 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-16T20:42:30.248 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-16T20:42:30.264 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:30.302 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking cephfs-mirror (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:30.501 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package socat. 2024-03-16T20:42:30.524 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-16T20:42:30.567 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-16T20:42:30.631 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-colorama. 2024-03-16T20:42:30.658 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-16T20:42:30.697 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-16T20:42:30.878 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package xmlstarlet. 2024-03-16T20:42:30.895 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-16T20:42:30.936 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-16T20:42:30.941 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package cephfs-shell. 2024-03-16T20:42:30.958 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:42:30.991 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking cephfs-shell (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:31.227 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package cephfs-top. 2024-03-16T20:42:31.244 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:42:31.286 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking cephfs-top (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:31.291 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-test. 2024-03-16T20:42:31.319 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../099-ceph-test_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:31.358 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-test (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:31.547 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-16T20:42:31.575 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:31.614 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libcephfs-dev (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:31.900 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-16T20:42:31.928 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-16T20:42:31.958 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-16T20:42:32.305 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-16T20:42:32.332 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-16T20:42:32.380 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-16T20:42:32.760 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package nvme-cli. 2024-03-16T20:42:32.787 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-16T20:42:32.826 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-16T20:42:33.249 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package pkg-config. 2024-03-16T20:42:33.276 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-16T20:42:33.315 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-16T20:42:33.526 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-16T20:42:33.542 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-16T20:42:33.576 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-16T20:42:33.770 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ceph-volume. 2024-03-16T20:42:33.798 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:42:33.845 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ceph-volume (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:33.982 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-16T20:42:33.996 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-16T20:42:34.029 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-16T20:42:34.191 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package cephfs-mirror. 2024-03-16T20:42:34.218 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:34.257 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking cephfs-mirror (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:34.299 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pastescript. 2024-03-16T20:42:34.327 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-16T20:42:34.366 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-16T20:42:34.635 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-colorama. 2024-03-16T20:42:34.663 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-03-16T20:42:34.686 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pluggy. 2024-03-16T20:42:34.710 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-03-16T20:42:34.714 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-16T20:42:34.752 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-16T20:42:35.046 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package cephfs-shell. 2024-03-16T20:42:35.048 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-psutil. 2024-03-16T20:42:35.073 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:42:35.076 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-16T20:42:35.112 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking cephfs-shell (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:35.114 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-16T20:42:35.374 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package cephfs-top. 2024-03-16T20:42:35.401 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-1681-gaba3c1d2-1jammy_all.deb ... 2024-03-16T20:42:35.440 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking cephfs-top (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:35.459 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-py. 2024-03-16T20:42:35.487 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-16T20:42:35.534 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-16T20:42:35.693 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-16T20:42:35.708 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:35.750 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libcephfs-dev (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:35.880 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pygments. 2024-03-16T20:42:35.908 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-16T20:42:35.946 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-16T20:42:36.120 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-16T20:42:36.148 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-16T20:42:36.195 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-16T20:42:36.434 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-16T20:42:36.462 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-16T20:42:36.500 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-16T20:42:36.542 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-16T20:42:36.569 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-03-16T20:42:36.608 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-16T20:42:36.795 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-toml. 2024-03-16T20:42:36.823 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-16T20:42:36.862 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-16T20:42:36.920 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package nvme-cli. 2024-03-16T20:42:36.948 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-16T20:42:36.978 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-16T20:42:37.123 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pytest. 2024-03-16T20:42:37.151 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-16T20:42:37.190 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-16T20:42:37.367 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package pkg-config. 2024-03-16T20:42:37.383 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-16T20:42:37.416 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-16T20:42:37.552 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-simplejson. 2024-03-16T20:42:37.580 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-16T20:42:37.618 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-16T20:42:37.678 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-16T20:42:37.705 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-16T20:42:37.744 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-16T20:42:37.889 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-16T20:42:37.916 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-16T20:42:37.955 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-16T20:42:38.173 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-16T20:42:38.200 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-16T20:42:38.239 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-16T20:42:38.542 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pastescript. 2024-03-16T20:42:38.569 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-03-16T20:42:38.608 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-16T20:42:38.693 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package radosgw. 2024-03-16T20:42:38.721 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../122-radosgw_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:38.759 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking radosgw (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:38.928 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pluggy. 2024-03-16T20:42:38.956 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-16T20:42:38.994 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-16T20:42:39.289 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-psutil. 2024-03-16T20:42:39.305 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-16T20:42:39.339 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-16T20:42:39.684 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-py. 2024-03-16T20:42:39.711 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-03-16T20:42:39.758 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-16T20:42:39.779 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package rbd-fuse. 2024-03-16T20:42:39.796 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:39.829 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking rbd-fuse (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:40.070 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pygments. 2024-03-16T20:42:40.097 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-16T20:42:40.126 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-16T20:42:40.136 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-16T20:42:40.385 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-16T20:42:40.503 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-16T20:42:40.574 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-16T20:42:40.601 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-16T20:42:40.632 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-16T20:42:40.735 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-16T20:42:40.943 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-toml. 2024-03-16T20:42:40.970 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-03-16T20:42:41.034 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-16T20:42:41.056 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-16T20:42:41.337 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-pytest. 2024-03-16T20:42:41.365 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-16T20:42:41.403 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-16T20:42:41.491 INFO:teuthology.orchestra.run.smithi122.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-16T20:42:41.555 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up cephadm (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:41.766 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package python3-simplejson. 2024-03-16T20:42:41.793 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-16T20:42:41.832 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-16T20:42:42.098 INFO:teuthology.orchestra.run.smithi122.stdout:Adding system user cephadm....done 2024-03-16T20:42:42.101 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-16T20:42:42.117 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-16T20:42:42.160 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-16T20:42:42.212 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-16T20:42:42.472 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-16T20:42:42.732 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-16T20:42:42.858 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-16T20:42:42.872 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package radosgw. 2024-03-16T20:42:42.900 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../122-radosgw_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:42.938 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking radosgw (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:43.109 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-16T20:42:43.377 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-16T20:42:43.478 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-16T20:42:43.725 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package rbd-fuse. 2024-03-16T20:42:43.741 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-1681-gaba3c1d2-1jammy_amd64.deb ... 2024-03-16T20:42:43.763 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-16T20:42:43.783 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking rbd-fuse (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:44.088 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-16T20:42:44.124 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-16T20:42:44.338 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-16T20:42:44.384 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-16T20:42:44.472 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-16T20:42:44.635 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-16T20:42:44.734 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-16T20:42:44.887 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-16T20:42:45.102 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-16T20:42:45.122 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-16T20:42:45.373 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-16T20:42:45.550 INFO:teuthology.orchestra.run.smithi195.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-16T20:42:45.616 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up cephadm (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:45.633 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-16T20:42:45.893 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-ceph-argparse (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:46.158 INFO:teuthology.orchestra.run.smithi195.stdout:Adding system user cephadm....done 2024-03-16T20:42:46.277 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-16T20:42:46.297 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-16T20:42:46.378 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-16T20:42:46.479 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-16T20:42:46.573 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-16T20:42:46.739 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-16T20:42:46.824 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-16T20:42:46.867 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-16T20:42:46.950 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-16T20:42:47.126 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-16T20:42:47.210 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-16T20:42:47.469 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-16T20:42:47.494 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-16T20:42:47.523 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-16T20:42:47.571 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-16T20:42:47.579 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-16T20:42:47.697 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-16T20:42:47.864 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-16T20:42:48.073 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-16T20:42:48.199 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-16T20:42:48.224 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-16T20:42:48.475 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-03-16T20:42:48.484 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-16T20:42:48.752 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-16T20:42:48.918 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-16T20:42:49.044 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up zip (3.0-12build2) ... 2024-03-16T20:42:49.162 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-16T20:42:49.202 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-16T20:42:49.463 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-16T20:42:49.706 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-16T20:42:49.789 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-16T20:42:50.023 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-16T20:42:50.048 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-16T20:42:50.175 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-16T20:42:50.274 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-ceph-argparse (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:50.309 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-16T20:42:50.542 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-16T20:42:50.593 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-16T20:42:50.668 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-16T20:42:50.786 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-03-16T20:42:50.969 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-16T20:42:51.028 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-16T20:42:51.138 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-16T20:42:51.204 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-16T20:42:51.399 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-16T20:42:51.547 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-16T20:42:51.797 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-16T20:42:51.822 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-16T20:42:51.831 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-16T20:42:51.877 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-16T20:42:51.987 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-16T20:42:52.132 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-16T20:42:52.338 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-16T20:42:52.464 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-16T20:42:52.467 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-16T20:42:52.719 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-16T20:42:52.732 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-16T20:42:52.853 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-ceph-common (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:53.008 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-16T20:42:53.121 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-idna (3.3-1) ... 2024-03-16T20:42:53.334 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up zip (3.0-12build2) ... 2024-03-16T20:42:53.413 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-16T20:42:53.460 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-16T20:42:54.129 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-16T20:42:54.371 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-16T20:42:54.522 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-16T20:42:54.536 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-16T20:42:54.649 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-16T20:42:54.762 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T20:42:54.888 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-16T20:42:54.958 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-16T20:42:55.331 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-16T20:42:55.351 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-16T20:42:55.457 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-16T20:42:55.583 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-16T20:42:55.593 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-16T20:42:55.893 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-16T20:42:55.945 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-16T20:42:56.144 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-16T20:42:56.230 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-03-16T20:42:56.370 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T20:42:56.479 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-16T20:42:56.539 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-16T20:42:56.772 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-16T20:42:56.883 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-16T20:42:57.057 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-16T20:42:57.133 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-16T20:42:57.260 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-ceph-common (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:42:57.324 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-16T20:42:57.552 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-idna (3.3-1) ... 2024-03-16T20:42:57.592 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-16T20:42:57.852 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-16T20:42:57.853 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-16T20:42:58.111 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-16T20:42:58.220 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-16T20:42:58.488 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-16T20:42:58.632 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-16T20:42:58.915 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-16T20:42:59.085 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-16T20:42:59.216 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-16T20:42:59.337 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T20:42:59.462 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-16T20:42:59.476 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-16T20:42:59.602 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-16T20:42:59.728 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-16T20:42:59.730 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-16T20:42:59.800 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-16T20:42:59.856 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-16T20:42:59.983 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-16T20:43:00.021 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-16T20:43:00.130 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-16T20:43:00.292 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-16T20:43:00.532 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-16T20:43:00.552 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-16T20:43:00.786 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T20:43:00.791 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-16T20:43:00.887 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-16T20:43:00.961 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-16T20:43:01.172 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-16T20:43:01.227 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T20:43:01.328 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-16T20:43:01.456 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-16T20:43:01.637 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-16T20:43:01.733 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-16T20:43:01.930 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-16T20:43:01.992 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-16T20:43:02.206 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-16T20:43:02.243 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-16T20:43:02.434 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-16T20:43:02.502 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-16T20:43:02.603 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-16T20:43:02.702 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-16T20:43:02.863 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-16T20:43:02.953 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-16T20:43:02.989 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-16T20:43:03.196 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-16T20:43:03.282 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-16T20:43:03.565 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-16T20:43:03.591 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-16T20:43:03.849 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-16T20:43:03.851 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-16T20:43:03.985 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-16T20:43:04.095 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-16T20:43:04.176 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-16T20:43:04.193 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-16T20:43:04.311 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librados2 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:04.412 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:04.413 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-16T20:43:04.513 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-16T20:43:04.539 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-16T20:43:04.841 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libcephfs2 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:04.940 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-16T20:43:04.975 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libradosstriper1 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:05.127 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-16T20:43:05.208 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-16T20:43:05.334 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-16T20:43:05.395 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librbd1 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:05.504 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-modules-core (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:05.594 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-16T20:43:05.630 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-fuse (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:05.703 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-16T20:43:05.858 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-16T20:43:05.858 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-16T20:43:06.004 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-16T20:43:06.314 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-16T20:43:06.381 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libcephfs-dev (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:06.507 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rados (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:06.582 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-16T20:43:06.633 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librgw2 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:06.759 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up cephfs-top (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:06.793 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-16T20:43:06.885 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-16T20:43:07.061 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-16T20:43:07.304 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-16T20:43:07.555 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-16T20:43:07.924 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-16T20:43:08.089 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rbd (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:08.208 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-16T20:43:08.215 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up rbd-fuse (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:08.341 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rgw (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:08.467 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cephfs (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:08.544 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-16T20:43:08.601 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up cephfs-shell (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:08.662 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librados2 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:08.727 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-common (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:08.771 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:08.897 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-16T20:43:09.234 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcephfs2 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:09.253 INFO:teuthology.orchestra.run.smithi122.stdout:Adding group ceph....done 2024-03-16T20:43:09.359 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libradosstriper1 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:09.477 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-16T20:43:09.647 INFO:teuthology.orchestra.run.smithi122.stdout:Adding system user ceph....done 2024-03-16T20:43:09.737 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librbd1 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:09.789 INFO:teuthology.orchestra.run.smithi122.stdout:Setting system user ceph properties....done 2024-03-16T20:43:09.800 INFO:teuthology.orchestra.run.smithi122.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-16T20:43:09.863 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-modules-core (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:09.933 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-16T20:43:09.989 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-fuse (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:10.198 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-16T20:43:10.198 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-16T20:43:10.255 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-16T20:43:10.674 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libcephfs-dev (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:10.800 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rados (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:10.926 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up librgw2 (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:10.946 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up cephfs-mirror (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:11.052 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up cephfs-top (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:11.144 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-16T20:43:11.145 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-16T20:43:11.161 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-16T20:43:11.597 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-test (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:11.723 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up radosgw (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:12.200 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-16T20:43:12.201 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-16T20:43:12.382 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rbd (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:12.516 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up rbd-fuse (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:12.641 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-base (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:12.642 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-rgw (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:12.768 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up python3-cephfs (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:12.894 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up cephfs-shell (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:12.976 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-16T20:43:12.995 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-common (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:13.435 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mds (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:13.547 INFO:teuthology.orchestra.run.smithi195.stdout:Adding group ceph....done 2024-03-16T20:43:13.639 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-16T20:43:13.639 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-16T20:43:13.924 INFO:teuthology.orchestra.run.smithi195.stdout:Adding system user ceph....done 2024-03-16T20:43:14.050 INFO:teuthology.orchestra.run.smithi195.stdout:Setting system user ceph properties....done 2024-03-16T20:43:14.064 INFO:teuthology.orchestra.run.smithi195.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-16T20:43:14.120 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:14.196 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-16T20:43:14.320 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-16T20:43:14.320 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-16T20:43:14.505 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-16T20:43:14.779 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-osd (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:15.005 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up cephfs-mirror (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:15.042 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-16T20:43:15.042 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-16T20:43:15.208 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-16T20:43:15.208 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-03-16T20:43:15.572 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:15.648 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-test (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:15.774 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up radosgw (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:15.843 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:15.994 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mon (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:16.211 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-16T20:43:16.212 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-16T20:43:16.221 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-16T20:43:16.221 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-16T20:43:16.662 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-cephadm (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:16.676 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-base (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:16.813 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:16.939 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-dashboard (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:17.022 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-16T20:43:17.073 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-volume (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:17.418 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-16T20:43:17.478 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mds (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:17.544 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-16T20:43:17.695 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-16T20:43:17.695 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-16T20:43:17.696 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-16T20:43:18.138 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:18.350 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-16T20:43:18.350 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-16T20:43:18.772 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-16T20:43:18.806 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-osd (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:19.068 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-16T20:43:19.068 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-16T20:43:19.408 INFO:teuthology.orchestra.run.smithi122.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-16T20:43:19.411 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-16T20:43:19.511 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-03-16T20:43:19.527 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:19.707 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-03-16T20:43:19.708 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-03-16T20:43:19.740 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:19.880 INFO:teuthology.orchestra.run.smithi122.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-16T20:43:19.881 INFO:teuthology.orchestra.run.smithi122.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-16T20:43:19.881 INFO:teuthology.orchestra.run.smithi122.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2024-03-16T20:43:19.881 INFO:teuthology.orchestra.run.smithi122.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2024-03-16T20:43:19.881 INFO:teuthology.orchestra.run.smithi122.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-16T20:43:19.881 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-03-16T20:43:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-16T20:43:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-16T20:43:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-16T20:43:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-16T20:43:19.881 INFO:teuthology.orchestra.run.smithi122.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-16T20:43:19.881 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-16T20:43:19.882 INFO:teuthology.orchestra.run.smithi122.stdout:The following additional packages will be installed: 2024-03-16T20:43:19.882 INFO:teuthology.orchestra.run.smithi122.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-16T20:43:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-16T20:43:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2024-03-16T20:43:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-16T20:43:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-16T20:43:19.883 INFO:teuthology.orchestra.run.smithi122.stdout:Suggested packages: 2024-03-16T20:43:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-16T20:43:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2024-03-16T20:43:19.883 INFO:teuthology.orchestra.run.smithi122.stdout: libengine-pkcs11-openssl 2024-03-16T20:43:19.921 INFO:teuthology.orchestra.run.smithi122.stdout:The following NEW packages will be installed: 2024-03-16T20:43:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-16T20:43:19.922 INFO:teuthology.orchestra.run.smithi122.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-16T20:43:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-16T20:43:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-16T20:43:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-16T20:43:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-16T20:43:19.923 INFO:teuthology.orchestra.run.smithi122.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-16T20:43:19.924 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages will be upgraded: 2024-03-16T20:43:19.924 INFO:teuthology.orchestra.run.smithi122.stdout: libssl3 2024-03-16T20:43:19.991 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mon (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:20.104 INFO:teuthology.orchestra.run.smithi122.stdout:1 upgraded, 39 newly installed, 0 to remove and 283 not upgraded. 2024-03-16T20:43:20.104 INFO:teuthology.orchestra.run.smithi122.stdout:Need to get 54.6 MB of archives. 2024-03-16T20:43:20.104 INFO:teuthology.orchestra.run.smithi122.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-16T20:43:20.104 INFO:teuthology.orchestra.run.smithi122.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-16T20:43:20.262 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-16T20:43:20.262 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-16T20:43:20.590 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-16T20:43:20.730 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-cephadm (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:20.848 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-16T20:43:20.873 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:20.887 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-16T20:43:20.893 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-16T20:43:20.898 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-16T20:43:20.903 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-16T20:43:20.910 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-16T20:43:20.916 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-16T20:43:20.925 INFO:teuthology.orchestra.run.smithi122.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-16T20:43:21.008 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-mgr-dashboard (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:21.100 INFO:teuthology.orchestra.run.smithi122.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-16T20:43:21.100 INFO:teuthology.orchestra.run.smithi122.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-16T20:43:21.158 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ceph-volume (19.0.0-1681-gaba3c1d2-1jammy) ... 2024-03-16T20:43:21.345 INFO:teuthology.orchestra.run.smithi122.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-16T20:43:21.503 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-16T20:43:21.612 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-16T20:43:21.739 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-16T20:43:22.457 INFO:teuthology.orchestra.run.smithi122.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-16T20:43:22.461 INFO:teuthology.orchestra.run.smithi122.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-16T20:43:22.461 INFO:teuthology.orchestra.run.smithi122.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-16T20:43:22.467 INFO:teuthology.orchestra.run.smithi122.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-16T20:43:22.523 INFO:teuthology.orchestra.run.smithi122.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-16T20:43:22.533 INFO:teuthology.orchestra.run.smithi122.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-16T20:43:22.537 INFO:teuthology.orchestra.run.smithi122.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-16T20:43:22.539 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-16T20:43:22.554 INFO:teuthology.orchestra.run.smithi122.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-16T20:43:22.719 INFO:teuthology.orchestra.run.smithi122.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-16T20:43:23.164 INFO:teuthology.orchestra.run.smithi195.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-16T20:43:23.167 DEBUG:teuthology.orchestra.run.smithi195:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-03-16T20:43:23.220 INFO:teuthology.orchestra.run.smithi195.stdout:Reading package lists... 2024-03-16T20:43:23.415 INFO:teuthology.orchestra.run.smithi195.stdout:Building dependency tree... 2024-03-16T20:43:23.416 INFO:teuthology.orchestra.run.smithi195.stdout:Reading state information... 2024-03-16T20:43:23.441 INFO:teuthology.orchestra.run.smithi122.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-16T20:43:23.442 INFO:teuthology.orchestra.run.smithi122.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-16T20:43:23.468 INFO:teuthology.orchestra.run.smithi122.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-16T20:43:23.546 INFO:teuthology.orchestra.run.smithi122.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-16T20:43:23.554 INFO:teuthology.orchestra.run.smithi122.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-16T20:43:23.556 INFO:teuthology.orchestra.run.smithi122.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-16T20:43:23.557 INFO:teuthology.orchestra.run.smithi122.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-16T20:43:23.558 INFO:teuthology.orchestra.run.smithi122.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-16T20:43:23.584 INFO:teuthology.orchestra.run.smithi122.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-16T20:43:23.588 INFO:teuthology.orchestra.run.smithi122.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-16T20:43:23.592 INFO:teuthology.orchestra.run.smithi195.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-03-16T20:43:23.593 INFO:teuthology.orchestra.run.smithi195.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-03-16T20:43:23.593 INFO:teuthology.orchestra.run.smithi195.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2024-03-16T20:43:23.593 INFO:teuthology.orchestra.run.smithi195.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2024-03-16T20:43:23.593 INFO:teuthology.orchestra.run.smithi195.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-03-16T20:43:23.593 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages were automatically installed and are no longer required: 2024-03-16T20:43:23.593 INFO:teuthology.orchestra.run.smithi195.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-16T20:43:23.593 INFO:teuthology.orchestra.run.smithi195.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-16T20:43:23.593 INFO:teuthology.orchestra.run.smithi195.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-16T20:43:23.594 INFO:teuthology.orchestra.run.smithi195.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-16T20:43:23.594 INFO:teuthology.orchestra.run.smithi195.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-16T20:43:23.594 INFO:teuthology.orchestra.run.smithi195.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-16T20:43:23.594 INFO:teuthology.orchestra.run.smithi195.stdout:The following additional packages will be installed: 2024-03-16T20:43:23.595 INFO:teuthology.orchestra.run.smithi195.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-03-16T20:43:23.595 INFO:teuthology.orchestra.run.smithi195.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-03-16T20:43:23.595 INFO:teuthology.orchestra.run.smithi195.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2024-03-16T20:43:23.595 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-03-16T20:43:23.595 INFO:teuthology.orchestra.run.smithi195.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-16T20:43:23.596 INFO:teuthology.orchestra.run.smithi195.stdout:Suggested packages: 2024-03-16T20:43:23.597 INFO:teuthology.orchestra.run.smithi195.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-03-16T20:43:23.597 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2024-03-16T20:43:23.597 INFO:teuthology.orchestra.run.smithi195.stdout: libengine-pkcs11-openssl 2024-03-16T20:43:23.634 INFO:teuthology.orchestra.run.smithi195.stdout:The following NEW packages will be installed: 2024-03-16T20:43:23.635 INFO:teuthology.orchestra.run.smithi195.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-03-16T20:43:23.635 INFO:teuthology.orchestra.run.smithi195.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-03-16T20:43:23.635 INFO:teuthology.orchestra.run.smithi195.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-03-16T20:43:23.636 INFO:teuthology.orchestra.run.smithi195.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-03-16T20:43:23.636 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-03-16T20:43:23.636 INFO:teuthology.orchestra.run.smithi195.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-03-16T20:43:23.636 INFO:teuthology.orchestra.run.smithi195.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-03-16T20:43:23.637 INFO:teuthology.orchestra.run.smithi195.stdout:The following packages will be upgraded: 2024-03-16T20:43:23.637 INFO:teuthology.orchestra.run.smithi195.stdout: libssl3 2024-03-16T20:43:23.739 INFO:teuthology.orchestra.run.smithi195.stdout:1 upgraded, 39 newly installed, 0 to remove and 283 not upgraded. 2024-03-16T20:43:23.739 INFO:teuthology.orchestra.run.smithi195.stdout:Need to get 54.6 MB of archives. 2024-03-16T20:43:23.739 INFO:teuthology.orchestra.run.smithi195.stdout:After this operation, 202 MB of additional disk space will be used. 2024-03-16T20:43:23.739 INFO:teuthology.orchestra.run.smithi195.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-03-16T20:43:23.910 INFO:teuthology.orchestra.run.smithi195.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-03-16T20:43:24.016 INFO:teuthology.orchestra.run.smithi195.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-03-16T20:43:24.033 INFO:teuthology.orchestra.run.smithi195.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-03-16T20:43:24.035 INFO:teuthology.orchestra.run.smithi195.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-03-16T20:43:24.039 INFO:teuthology.orchestra.run.smithi195.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-03-16T20:43:24.041 INFO:teuthology.orchestra.run.smithi195.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-03-16T20:43:24.042 INFO:teuthology.orchestra.run.smithi195.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-03-16T20:43:24.044 INFO:teuthology.orchestra.run.smithi195.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-03-16T20:43:24.046 INFO:teuthology.orchestra.run.smithi195.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-03-16T20:43:24.047 INFO:teuthology.orchestra.run.smithi195.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-03-16T20:43:24.060 INFO:teuthology.orchestra.run.smithi195.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-03-16T20:43:24.074 INFO:teuthology.orchestra.run.smithi195.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-03-16T20:43:24.306 INFO:teuthology.orchestra.run.smithi122.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-16T20:43:24.389 INFO:teuthology.orchestra.run.smithi122.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-16T20:43:24.389 INFO:teuthology.orchestra.run.smithi122.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-16T20:43:24.390 INFO:teuthology.orchestra.run.smithi122.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-16T20:43:24.479 INFO:teuthology.orchestra.run.smithi122.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-16T20:43:24.485 INFO:teuthology.orchestra.run.smithi122.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-16T20:43:24.573 INFO:teuthology.orchestra.run.smithi195.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-03-16T20:43:24.576 INFO:teuthology.orchestra.run.smithi195.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-03-16T20:43:24.634 INFO:teuthology.orchestra.run.smithi195.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-03-16T20:43:24.639 INFO:teuthology.orchestra.run.smithi195.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-03-16T20:43:24.639 INFO:teuthology.orchestra.run.smithi195.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-03-16T20:43:24.647 INFO:teuthology.orchestra.run.smithi195.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-03-16T20:43:24.650 INFO:teuthology.orchestra.run.smithi195.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-03-16T20:43:24.662 INFO:teuthology.orchestra.run.smithi195.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-03-16T20:43:24.663 INFO:teuthology.orchestra.run.smithi195.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-03-16T20:43:24.697 INFO:teuthology.orchestra.run.smithi195.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-03-16T20:43:24.697 INFO:teuthology.orchestra.run.smithi195.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-03-16T20:43:24.729 INFO:teuthology.orchestra.run.smithi195.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-03-16T20:43:24.762 INFO:teuthology.orchestra.run.smithi195.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-03-16T20:43:24.829 INFO:teuthology.orchestra.run.smithi195.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-03-16T20:43:24.830 INFO:teuthology.orchestra.run.smithi195.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-03-16T20:43:24.831 INFO:teuthology.orchestra.run.smithi195.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-03-16T20:43:24.832 INFO:teuthology.orchestra.run.smithi195.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-03-16T20:43:24.849 INFO:teuthology.orchestra.run.smithi195.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-03-16T20:43:24.852 INFO:teuthology.orchestra.run.smithi195.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-03-16T20:43:24.941 INFO:teuthology.orchestra.run.smithi122.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-16T20:43:24.943 INFO:teuthology.orchestra.run.smithi122.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-16T20:43:25.190 INFO:teuthology.orchestra.run.smithi195.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-03-16T20:43:25.191 INFO:teuthology.orchestra.run.smithi195.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-03-16T20:43:25.191 INFO:teuthology.orchestra.run.smithi195.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-03-16T20:43:25.191 INFO:teuthology.orchestra.run.smithi195.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-03-16T20:43:25.191 INFO:teuthology.orchestra.run.smithi195.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-03-16T20:43:25.191 INFO:teuthology.orchestra.run.smithi195.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-03-16T20:43:25.209 INFO:teuthology.orchestra.run.smithi195.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-03-16T20:43:25.210 INFO:teuthology.orchestra.run.smithi195.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-03-16T20:43:25.351 INFO:teuthology.orchestra.run.smithi122.stdout:Preconfiguring packages ... 2024-03-16T20:43:25.526 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 54.6 MB in 5s (10.9 MB/s) 2024-03-16T20:43:25.615 INFO:teuthology.orchestra.run.smithi195.stdout:Preconfiguring packages ... 2024-03-16T20:43:25.664 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package flex. 2024-03-16T20:43:25.755 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173657 files and directories currently installed.) 2024-03-16T20:43:25.761 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-16T20:43:25.785 INFO:teuthology.orchestra.run.smithi195.stdout:Fetched 54.6 MB in 2s (34.9 MB/s) 2024-03-16T20:43:25.804 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-16T20:43:25.930 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package flex. 2024-03-16T20:43:26.033 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173657 files and directories currently installed.) 2024-03-16T20:43:26.038 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-03-16T20:43:26.078 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking flex (2.6.4-8build2) ... 2024-03-16T20:43:26.379 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-16T20:43:26.535 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-16T20:43:26.628 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-16T20:43:26.800 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-03-16T20:43:26.920 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-16T20:43:27.211 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-16T20:43:27.461 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package bison. 2024-03-16T20:43:27.566 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173730 files and directories currently installed.) 2024-03-16T20:43:27.571 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-16T20:43:27.622 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-16T20:43:27.743 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package bison. 2024-03-16T20:43:27.845 INFO:teuthology.orchestra.run.smithi195.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173730 files and directories currently installed.) 2024-03-16T20:43:27.851 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-03-16T20:43:27.895 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-03-16T20:43:28.030 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-16T20:43:28.058 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-16T20:43:28.097 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-16T20:43:28.321 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-03-16T20:43:28.349 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-03-16T20:43:28.388 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-03-16T20:43:28.417 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package dump. 2024-03-16T20:43:28.444 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-16T20:43:28.483 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking dump (0.4b47-3) ... 2024-03-16T20:43:28.674 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package dump. 2024-03-16T20:43:28.701 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-03-16T20:43:28.732 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking dump (0.4b47-3) ... 2024-03-16T20:43:28.753 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package indent. 2024-03-16T20:43:28.781 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-16T20:43:28.820 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-16T20:43:29.011 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package indent. 2024-03-16T20:43:29.038 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-03-16T20:43:29.077 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-16T20:43:29.089 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-16T20:43:29.116 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-16T20:43:29.155 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-16T20:43:29.338 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-03-16T20:43:29.366 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-03-16T20:43:29.404 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-16T20:43:29.476 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-16T20:43:29.503 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-16T20:43:29.542 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-16T20:43:29.820 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-16T20:43:29.836 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-16T20:43:29.870 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-16T20:43:30.215 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-16T20:43:30.242 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-16T20:43:30.281 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-16T20:43:30.335 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-03-16T20:43:30.363 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-03-16T20:43:30.402 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-16T20:43:30.534 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-16T20:43:30.549 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-16T20:43:30.601 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-16T20:43:30.671 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-03-16T20:43:30.699 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-03-16T20:43:30.737 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-03-16T20:43:30.853 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libjson-perl. 2024-03-16T20:43:30.869 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-16T20:43:30.903 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-16T20:43:31.116 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libfl2:amd64. 2024-03-16T20:43:31.144 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-03-16T20:43:31.182 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-03-16T20:43:31.222 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-16T20:43:31.250 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-16T20:43:31.289 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-16T20:43:31.435 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-03-16T20:43:31.451 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-03-16T20:43:31.485 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-16T20:43:31.704 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libjson-perl. 2024-03-16T20:43:31.722 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-03-16T20:43:31.762 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-03-16T20:43:32.074 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libllvm14:amd64. 2024-03-16T20:43:32.102 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-03-16T20:43:32.140 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-16T20:43:32.983 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-16T20:43:33.011 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-16T20:43:33.050 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-16T20:43:33.328 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-16T20:43:33.355 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-16T20:43:33.394 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-16T20:43:33.655 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-16T20:43:33.683 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-16T20:43:33.738 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-16T20:43:33.818 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-03-16T20:43:33.846 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-16T20:43:33.884 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-16T20:43:34.083 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-16T20:43:34.110 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-16T20:43:34.137 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libmbim-proxy. 2024-03-16T20:43:34.149 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-16T20:43:34.153 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-03-16T20:43:34.187 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-16T20:43:34.461 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-16T20:43:34.488 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-16T20:43:34.489 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-03-16T20:43:34.517 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-16T20:43:34.527 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-16T20:43:34.556 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-16T20:43:34.821 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-16T20:43:34.837 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-16T20:43:34.871 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T20:43:34.909 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libndp0:amd64. 2024-03-16T20:43:34.937 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-03-16T20:43:34.976 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-16T20:43:35.199 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-16T20:43:35.227 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-16T20:43:35.253 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libnm0:amd64. 2024-03-16T20:43:35.265 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-16T20:43:35.282 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-03-16T20:43:35.320 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-16T20:43:35.577 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-16T20:43:35.593 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-16T20:43:35.635 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-16T20:43:35.723 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libpq5:amd64. 2024-03-16T20:43:35.751 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-16T20:43:35.790 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T20:43:35.879 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-16T20:43:35.895 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-16T20:43:35.929 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-16T20:43:36.168 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-03-16T20:43:36.192 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-16T20:43:36.234 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-16T20:43:36.474 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-16T20:43:36.501 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-16T20:43:36.520 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libqmi-proxy. 2024-03-16T20:43:36.548 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-03-16T20:43:36.578 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-16T20:43:36.651 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-16T20:43:36.822 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-03-16T20:43:36.838 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-03-16T20:43:36.880 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-16T20:43:37.044 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package modemmanager. 2024-03-16T20:43:37.071 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-16T20:43:37.110 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-16T20:43:37.409 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-03-16T20:43:37.437 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-03-16T20:43:37.476 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-03-16T20:43:37.716 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package network-manager. 2024-03-16T20:43:37.744 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-16T20:43:37.745 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package modemmanager. 2024-03-16T20:43:37.773 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-03-16T20:43:37.803 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-16T20:43:37.815 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-16T20:43:38.457 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ppp. 2024-03-16T20:43:38.484 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-16T20:43:38.485 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package network-manager. 2024-03-16T20:43:38.512 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-03-16T20:43:38.559 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-03-16T20:43:38.755 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-16T20:43:39.036 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package pptp-linux. 2024-03-16T20:43:39.052 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-16T20:43:39.094 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-16T20:43:39.175 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package ppp. 2024-03-16T20:43:39.202 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-03-16T20:43:39.297 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-16T20:43:39.313 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-16T20:43:39.355 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-16T20:43:39.484 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-03-16T20:43:39.658 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-16T20:43:39.674 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-16T20:43:39.716 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-client-common (238) ... 2024-03-16T20:43:39.796 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package pptp-linux. 2024-03-16T20:43:39.824 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-03-16T20:43:39.862 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-03-16T20:43:40.020 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-16T20:43:40.048 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-16T20:43:40.086 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T20:43:40.090 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package network-manager-pptp. 2024-03-16T20:43:40.106 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-03-16T20:43:40.140 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-03-16T20:43:40.428 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-client-common. 2024-03-16T20:43:40.456 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-03-16T20:43:40.495 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-client-common (238) ... 2024-03-16T20:43:40.691 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-common. 2024-03-16T20:43:40.720 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-16T20:43:40.777 INFO:teuthology.orchestra.run.smithi122.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-16T20:43:40.805 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-common (238) ... 2024-03-16T20:43:40.841 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-client-14. 2024-03-16T20:43:40.869 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-16T20:43:40.908 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T20:43:41.251 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-14. 2024-03-16T20:43:41.279 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-16T20:43:41.320 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T20:43:41.471 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-common. 2024-03-16T20:43:41.500 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-03-16T20:43:41.548 INFO:teuthology.orchestra.run.smithi195.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-03-16T20:43:41.577 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-common (238) ... 2024-03-16T20:43:42.029 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-14. 2024-03-16T20:43:42.057 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-03-16T20:43:42.098 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T20:43:42.670 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql. 2024-03-16T20:43:42.699 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-16T20:43:42.745 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql (14+238) ... 2024-03-16T20:43:42.981 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-client. 2024-03-16T20:43:42.997 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-16T20:43:43.039 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-client (14+238) ... 2024-03-16T20:43:43.250 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-16T20:43:43.266 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-16T20:43:43.308 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-16T20:43:43.498 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql. 2024-03-16T20:43:43.527 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-03-16T20:43:43.565 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql (14+238) ... 2024-03-16T20:43:43.627 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package tcl8.6. 2024-03-16T20:43:43.655 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-16T20:43:43.694 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-16T20:43:43.809 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-client. 2024-03-16T20:43:43.837 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-03-16T20:43:43.875 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-client (14+238) ... 2024-03-16T20:43:43.988 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package tcl. 2024-03-16T20:43:44.005 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-16T20:43:44.051 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-16T20:43:44.128 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package postgresql-contrib. 2024-03-16T20:43:44.156 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-03-16T20:43:44.194 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking postgresql-contrib (14+238) ... 2024-03-16T20:43:44.366 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package wpasupplicant. 2024-03-16T20:43:44.394 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-16T20:43:44.424 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-16T20:43:44.488 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package tcl8.6. 2024-03-16T20:43:44.516 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-03-16T20:43:44.546 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-16T20:43:44.779 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-16T20:43:44.807 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-16T20:43:44.824 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package tcl. 2024-03-16T20:43:44.845 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-16T20:43:44.852 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-03-16T20:43:44.895 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-03-16T20:43:45.223 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-16T20:43:45.244 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package wpasupplicant. 2024-03-16T20:43:45.251 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-16T20:43:45.272 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-03-16T20:43:45.299 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-16T20:43:45.319 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-16T20:43:45.594 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-client-common (238) ... 2024-03-16T20:43:45.690 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-03-16T20:43:45.706 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-03-16T20:43:45.748 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-03-16T20:43:45.780 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-16T20:43:45.897 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-16T20:43:46.048 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T20:43:46.076 INFO:teuthology.orchestra.run.smithi195.stdout:Selecting previously unselected package usb-modeswitch. 2024-03-16T20:43:46.104 INFO:teuthology.orchestra.run.smithi195.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-03-16T20:43:46.142 INFO:teuthology.orchestra.run.smithi195.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-16T20:43:46.157 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-16T20:43:46.258 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-16T20:43:46.376 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T20:43:46.413 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-client-common (238) ... 2024-03-16T20:43:46.715 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up flex (2.6.4-8build2) ... 2024-03-16T20:43:46.843 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-03-16T20:43:46.867 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-16T20:43:46.957 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-16T20:43:46.969 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T20:43:47.095 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-03-16T20:43:47.221 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-03-16T20:43:47.339 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T20:43:47.470 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-16T20:43:47.470 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-16T20:43:47.827 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-03-16T20:43:47.911 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-03-16T20:43:47.968 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-16T20:43:48.094 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-16T20:43:48.220 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-16T20:43:48.322 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-16T20:43:48.322 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-03-16T20:43:48.346 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-16T20:43:48.472 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-16T20:43:48.598 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-16T20:43:48.708 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-16T20:43:48.773 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-03-16T20:43:48.874 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-03-16T20:43:48.975 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-03-16T20:43:49.101 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-03-16T20:43:49.227 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-03-16T20:43:49.330 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-16T20:43:49.361 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-03-16T20:43:49.456 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up dump (0.4b47-3) ... 2024-03-16T20:43:49.487 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-03-16T20:43:49.549 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-16T20:43:49.615 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-16T20:43:49.733 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-16T20:43:49.835 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-16T20:43:49.952 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-16T20:43:50.068 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-03-16T20:43:50.100 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-16T20:43:50.194 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-16T20:43:50.194 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up dump (0.4b47-3) ... 2024-03-16T20:43:50.263 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-16T20:43:50.288 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-03-16T20:43:50.354 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-03-16T20:43:50.389 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-16T20:43:50.480 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-03-16T20:43:50.582 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-16T20:43:50.708 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-16T20:43:50.714 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up tcl (8.6.11+1build2) ... 2024-03-16T20:43:50.816 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libjson-perl (4.04000-1) ... 2024-03-16T20:43:50.817 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-16T20:43:50.917 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-03-16T20:43:50.960 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-client (14+238) ... 2024-03-16T20:43:50.990 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-03-16T20:43:51.051 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-03-16T20:43:51.061 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-16T20:43:51.178 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-03-16T20:43:51.404 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-03-16T20:43:51.507 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:43:51.507 INFO:teuthology.orchestra.run.smithi122.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-16T20:43:51.507 INFO:teuthology.orchestra.run.smithi122.stdout:which means they are currently configured by ifupdown: 2024-03-16T20:43:51.507 INFO:teuthology.orchestra.run.smithi122.stdout:- enp3s0f1 2024-03-16T20:43:51.507 INFO:teuthology.orchestra.run.smithi122.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-16T20:43:51.507 INFO:teuthology.orchestra.run.smithi122.stdout:remove their configuration from /etc/network/interfaces. 2024-03-16T20:43:51.507 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:43:51.530 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-03-16T20:43:51.629 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-16T20:43:51.656 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-03-16T20:43:51.791 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-client (14+238) ... 2024-03-16T20:43:51.917 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-03-16T20:43:51.943 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-16T20:43:52.265 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-16T20:43:52.386 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:43:52.386 INFO:teuthology.orchestra.run.smithi195.stdout:The following network interfaces were found in /etc/network/interfaces 2024-03-16T20:43:52.387 INFO:teuthology.orchestra.run.smithi195.stdout:which means they are currently configured by ifupdown: 2024-03-16T20:43:52.387 INFO:teuthology.orchestra.run.smithi195.stdout:- enp3s0f1 2024-03-16T20:43:52.387 INFO:teuthology.orchestra.run.smithi195.stdout:If you want to manage those interfaces with NetworkManager instead 2024-03-16T20:43:52.387 INFO:teuthology.orchestra.run.smithi195.stdout:remove their configuration from /etc/network/interfaces. 2024-03-16T20:43:52.387 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:43:52.501 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-03-16T20:43:52.756 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-16T20:43:52.822 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-03-16T20:43:52.891 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-16T20:43:53.017 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-16T20:43:53.125 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-03-16T20:43:53.143 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-common (238) ... 2024-03-16T20:43:53.636 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-03-16T20:43:53.771 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-03-16T20:43:53.897 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-03-16T20:43:53.990 INFO:teuthology.orchestra.run.smithi122.stdout:Adding user postgres to group ssl-cert 2024-03-16T20:43:54.023 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-common (238) ... 2024-03-16T20:43:54.214 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:43:54.214 INFO:teuthology.orchestra.run.smithi122.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-16T20:43:54.397 INFO:teuthology.orchestra.run.smithi122.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-16T20:43:54.398 INFO:teuthology.orchestra.run.smithi122.stdout:Removing obsolete dictionary files: 2024-03-16T20:43:54.940 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-16T20:43:54.945 INFO:teuthology.orchestra.run.smithi195.stdout:Adding user postgres to group ssl-cert 2024-03-16T20:43:55.179 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:43:55.179 INFO:teuthology.orchestra.run.smithi195.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-03-16T20:43:55.286 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T20:43:55.369 INFO:teuthology.orchestra.run.smithi195.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-03-16T20:43:55.369 INFO:teuthology.orchestra.run.smithi195.stdout:Removing obsolete dictionary files: 2024-03-16T20:43:55.589 INFO:teuthology.orchestra.run.smithi122.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-16T20:43:55.607 INFO:teuthology.orchestra.run.smithi122.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-16T20:43:55.656 INFO:teuthology.orchestra.run.smithi122.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-16T20:43:55.656 INFO:teuthology.orchestra.run.smithi122.stdout:This user must also own the server process. 2024-03-16T20:43:55.656 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:43:55.656 INFO:teuthology.orchestra.run.smithi122.stdout:The database cluster will be initialized with locale "en_US". 2024-03-16T20:43:55.656 INFO:teuthology.orchestra.run.smithi122.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-16T20:43:55.657 INFO:teuthology.orchestra.run.smithi122.stdout:The default text search configuration will be set to "english". 2024-03-16T20:43:55.657 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:43:55.657 INFO:teuthology.orchestra.run.smithi122.stdout:Data page checksums are disabled. 2024-03-16T20:43:55.657 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:43:55.657 INFO:teuthology.orchestra.run.smithi122.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-16T20:43:55.657 INFO:teuthology.orchestra.run.smithi122.stdout:creating subdirectories ... ok 2024-03-16T20:43:55.657 INFO:teuthology.orchestra.run.smithi122.stdout:selecting dynamic shared memory implementation ... posix 2024-03-16T20:43:55.671 INFO:teuthology.orchestra.run.smithi122.stdout:selecting default max_connections ... 100 2024-03-16T20:43:55.692 INFO:teuthology.orchestra.run.smithi122.stdout:selecting default shared_buffers ... 128MB 2024-03-16T20:43:55.702 INFO:teuthology.orchestra.run.smithi122.stdout:selecting default time zone ... Etc/UTC 2024-03-16T20:43:55.703 INFO:teuthology.orchestra.run.smithi122.stdout:creating configuration files ... ok 2024-03-16T20:43:55.817 INFO:teuthology.orchestra.run.smithi122.stdout:running bootstrap script ... ok 2024-03-16T20:43:55.917 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-03-16T20:43:56.299 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-03-16T20:43:56.414 INFO:teuthology.orchestra.run.smithi122.stdout:performing post-bootstrap initialization ... ok 2024-03-16T20:43:56.631 INFO:teuthology.orchestra.run.smithi195.stdout:Creating new PostgreSQL cluster 14/main ... 2024-03-16T20:43:56.651 INFO:teuthology.orchestra.run.smithi195.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-03-16T20:43:56.689 INFO:teuthology.orchestra.run.smithi195.stdout:The files belonging to this database system will be owned by user "postgres". 2024-03-16T20:43:56.689 INFO:teuthology.orchestra.run.smithi195.stdout:This user must also own the server process. 2024-03-16T20:43:56.689 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:43:56.689 INFO:teuthology.orchestra.run.smithi195.stdout:The database cluster will be initialized with locale "en_US". 2024-03-16T20:43:56.689 INFO:teuthology.orchestra.run.smithi195.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-03-16T20:43:56.689 INFO:teuthology.orchestra.run.smithi195.stdout:The default text search configuration will be set to "english". 2024-03-16T20:43:56.689 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:43:56.689 INFO:teuthology.orchestra.run.smithi195.stdout:Data page checksums are disabled. 2024-03-16T20:43:56.689 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:43:56.690 INFO:teuthology.orchestra.run.smithi195.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-03-16T20:43:56.690 INFO:teuthology.orchestra.run.smithi195.stdout:creating subdirectories ... ok 2024-03-16T20:43:56.690 INFO:teuthology.orchestra.run.smithi195.stdout:selecting dynamic shared memory implementation ... posix 2024-03-16T20:43:56.702 INFO:teuthology.orchestra.run.smithi195.stdout:selecting default max_connections ... 100 2024-03-16T20:43:56.725 INFO:teuthology.orchestra.run.smithi195.stdout:selecting default shared_buffers ... 128MB 2024-03-16T20:43:56.735 INFO:teuthology.orchestra.run.smithi195.stdout:selecting default time zone ... Etc/UTC 2024-03-16T20:43:56.735 INFO:teuthology.orchestra.run.smithi195.stdout:creating configuration files ... ok 2024-03-16T20:43:56.853 INFO:teuthology.orchestra.run.smithi195.stdout:running bootstrap script ... ok 2024-03-16T20:43:57.106 INFO:teuthology.orchestra.run.smithi122.stdout:syncing data to disk ... ok 2024-03-16T20:43:57.500 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-16T20:43:57.530 INFO:teuthology.orchestra.run.smithi195.stdout:performing post-bootstrap initialization ... ok 2024-03-16T20:43:58.186 INFO:teuthology.orchestra.run.smithi195.stdout:syncing data to disk ... ok 2024-03-16T20:43:58.607 INFO:teuthology.orchestra.run.smithi195.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-03-16T20:44:00.309 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-16T20:44:00.435 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-16T20:44:00.561 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up postgresql (14+238) ... 2024-03-16T20:44:00.696 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-16T20:44:00.813 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-16T20:44:01.052 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-16T20:44:01.052 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-16T20:44:01.365 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-03-16T20:44:01.474 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql-contrib (14+238) ... 2024-03-16T20:44:01.582 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-16T20:44:01.600 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up postgresql (14+238) ... 2024-03-16T20:44:01.726 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-03-16T20:44:01.860 INFO:teuthology.orchestra.run.smithi195.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-03-16T20:44:01.901 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-16T20:44:02.053 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-16T20:44:02.116 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-03-16T20:44:02.116 INFO:teuthology.orchestra.run.smithi195.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-03-16T20:44:02.671 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-03-16T20:44:02.998 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-16T20:44:03.142 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-16T20:44:03.323 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-16T20:44:03.422 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-16T20:44:03.904 INFO:teuthology.orchestra.run.smithi122.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-16T20:44:03.909 DEBUG:teuthology.parallel:result is None 2024-03-16T20:44:04.412 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-03-16T20:44:04.520 INFO:teuthology.orchestra.run.smithi195.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-16T20:44:05.043 INFO:teuthology.orchestra.run.smithi195.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-16T20:44:05.047 DEBUG:teuthology.parallel:result is None 2024-03-16T20:44:05.047 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-16T20:44:05.187 DEBUG:teuthology.orchestra.run.smithi122:> dpkg-query -W -f '${Version}' ceph 2024-03-16T20:44:05.207 INFO:teuthology.orchestra.run.smithi122.stdout:19.0.0-1681-gaba3c1d2-1jammy 2024-03-16T20:44:05.207 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1681-gaba3c1d2-1jammy 2024-03-16T20:44:05.208 INFO:teuthology.task.install:The correct ceph version 19.0.0-1681-gaba3c1d2-1jammy is installed. 2024-03-16T20:44:05.209 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b 2024-03-16T20:44:05.423 DEBUG:teuthology.orchestra.run.smithi195:> dpkg-query -W -f '${Version}' ceph 2024-03-16T20:44:05.442 INFO:teuthology.orchestra.run.smithi195.stdout:19.0.0-1681-gaba3c1d2-1jammy 2024-03-16T20:44:05.442 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1681-gaba3c1d2-1jammy 2024-03-16T20:44:05.442 INFO:teuthology.task.install:The correct ceph version 19.0.0-1681-gaba3c1d2-1jammy is installed. 2024-03-16T20:44:05.444 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-16T20:44:05.444 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:05.444 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T20:44:05.458 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:05.458 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-16T20:44:05.495 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-16T20:44:05.495 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:05.496 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T20:44:05.512 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T20:44:05.568 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:05.568 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/daemon-helper 2024-03-16T20:44:05.581 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-16T20:44:05.635 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-16T20:44:05.635 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:05.635 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T20:44:05.648 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T20:44:05.703 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:05.703 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-16T20:44:05.715 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-16T20:44:05.774 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-16T20:44:05.775 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:05.775 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T20:44:05.787 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T20:44:05.842 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:05.843 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/usr/bin/stdin-killer 2024-03-16T20:44:05.856 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-16T20:44:05.910 INFO:teuthology.run_tasks:Running task ceph... 2024-03-16T20:44:05.987 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-03-16T20:44:05.987 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /var/log/ceph 2024-03-16T20:44:05.990 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 777 /var/log/ceph 2024-03-16T20:44:06.002 INFO:tasks.ceph:Disabling ceph logrotate... 2024-03-16T20:44:06.002 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-16T20:44:06.046 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/logrotate.d/ceph 2024-03-16T20:44:06.058 INFO:tasks.ceph:Creating extra log directories... 2024-03-16T20:44:06.058 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-16T20:44:06.102 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-03-16T20:44:06.121 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-03-16T20:44:06.121 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-03-16T20:44:06.121 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-03-16T20:44:06.122 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:06.122 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-16T20:44:06.201 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-16T20:44:06.275 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:06.276 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-03-16T20:44:06.281 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-03-16T20:44:06.359 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-03-16T20:44:06.359 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-03-16T20:44:06.359 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602163', 'branch': 'wip-lusov-qdb-squid', 'description': 'fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell}', 'email': None, 'first_in_suite': False, 'job_id': '7602163', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-lusov-qdb-squid'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lusov-qdb-squid', 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b'}}, 'owner': 'scheduled_leonidus@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 6918, 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b', 'sleep_before_teardown': 0, 'subset': '111/10000', 'suite': 'fs', 'suite_branch': 'wip-lusov-qdb-squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi122.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEllsH28MeQ1SW8dxOYFtPgzrJX/rVVTPJCd3FSmYFmusoy8payGfi9Hln4nFDY1GMRB7WRpogcPhOEfv/LrjLw=', 'smithi195.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGktsF1WOWYs6IgkXWlub27PjDKbkq2fNseBYcLGWD6zOSTVzBaG1oqXQzKXi3cW65Xd617wzRsqSvsyiHVh6yc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_cephfs_shell']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': 'e691533f9cbb33d85b2187bba20d7102f098636d', 'timestamp': '2024-03-15_11:44:52', 'tube': 'smithi', 'user': 'leonidus', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-03-16T20:44:06.360 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-16T20:44:06.362 DEBUG:teuthology.orchestra.run.smithi195:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-03-16T20:44:06.403 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/run/ceph 2024-03-16T20:44:06.409 DEBUG:teuthology.orchestra.run.smithi195:> sudo install -d -m0777 -- /var/run/ceph 2024-03-16T20:44:06.459 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:06.459 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T20:44:06.468 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T20:44:06.468 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-03-16T20:44:06.517 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T20:44:06.517 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T20:44:06.517 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-03-16T20:44:06.517 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T20:44:06.517 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-03-16 20:37:56.774805176 +0000 2024-03-16T20:44:06.517 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-03-16 20:37:56.478807817 +0000 2024-03-16T20:44:06.517 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-03-16 20:37:56.478807817 +0000 2024-03-16T20:44:06.517 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-03-16T20:44:06.517 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T20:44:06.570 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-03-16T20:44:06.571 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-03-16T20:44:06.571 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.00032595 s, 1.6 MB/s 2024-03-16T20:44:06.572 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T20:44:06.622 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-03-16T20:44:06.668 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T20:44:06.668 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T20:44:06.668 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-03-16T20:44:06.669 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T20:44:06.669 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-03-16 20:37:56.990803250 +0000 2024-03-16T20:44:06.669 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-03-16 20:37:56.986803285 +0000 2024-03-16T20:44:06.669 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-03-16 20:37:56.986803285 +0000 2024-03-16T20:44:06.669 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-03-16T20:44:06.669 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T20:44:06.722 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-03-16T20:44:06.722 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-03-16T20:44:06.722 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000419379 s, 1.2 MB/s 2024-03-16T20:44:06.724 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T20:44:06.774 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-03-16T20:44:06.820 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T20:44:06.821 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T20:44:06.821 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 920 Links: 1 2024-03-16T20:44:06.821 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T20:44:06.821 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-03-16 20:37:57.722796720 +0000 2024-03-16T20:44:06.821 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-03-16 20:37:57.474798932 +0000 2024-03-16T20:44:06.821 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-03-16 20:37:57.474798932 +0000 2024-03-16T20:44:06.821 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-03-16T20:44:06.821 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T20:44:06.874 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-03-16T20:44:06.874 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-03-16T20:44:06.874 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000397939 s, 1.3 MB/s 2024-03-16T20:44:06.875 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T20:44:06.926 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-03-16T20:44:06.972 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T20:44:06.972 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T20:44:06.972 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 933 Links: 1 2024-03-16T20:44:06.972 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T20:44:06.972 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-03-16 20:37:58.226792223 +0000 2024-03-16T20:44:06.973 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-03-16 20:37:57.946794721 +0000 2024-03-16T20:44:06.973 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-03-16 20:37:57.946794721 +0000 2024-03-16T20:44:06.973 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-03-16T20:44:06.973 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T20:44:07.026 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-03-16T20:44:07.026 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-03-16T20:44:07.026 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000407177 s, 1.3 MB/s 2024-03-16T20:44:07.028 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T20:44:07.078 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-16T20:44:07.078 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:07.078 DEBUG:teuthology.orchestra.run.smithi195:> dd if=/scratch_devs of=/dev/stdout 2024-03-16T20:44:07.084 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-16T20:44:07.084 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_1 2024-03-16T20:44:07.132 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-16T20:44:07.132 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T20:44:07.132 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 882 Links: 1 2024-03-16T20:44:07.132 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T20:44:07.132 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-03-16 20:37:58.737174526 +0000 2024-03-16T20:44:07.132 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-03-16 20:37:58.733174563 +0000 2024-03-16T20:44:07.132 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-03-16 20:37:58.733174563 +0000 2024-03-16T20:44:07.132 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-03-16T20:44:07.132 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-16T20:44:07.186 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-03-16T20:44:07.186 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-03-16T20:44:07.186 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000285902 s, 1.8 MB/s 2024-03-16T20:44:07.187 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-16T20:44:07.237 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_2 2024-03-16T20:44:07.283 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-16T20:44:07.283 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T20:44:07.284 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-03-16T20:44:07.284 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T20:44:07.284 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-03-16 20:37:59.509167640 +0000 2024-03-16T20:44:07.284 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-03-16 20:37:59.237170067 +0000 2024-03-16T20:44:07.284 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-03-16 20:37:59.237170067 +0000 2024-03-16T20:44:07.284 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-03-16T20:44:07.284 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-16T20:44:07.337 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-03-16T20:44:07.337 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-03-16T20:44:07.337 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000307138 s, 1.7 MB/s 2024-03-16T20:44:07.339 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-16T20:44:07.388 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_3 2024-03-16T20:44:07.435 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-16T20:44:07.436 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T20:44:07.436 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 918 Links: 1 2024-03-16T20:44:07.436 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T20:44:07.436 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-03-16 20:38:00.001163251 +0000 2024-03-16T20:44:07.436 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-03-16 20:37:59.753165463 +0000 2024-03-16T20:44:07.436 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-03-16 20:37:59.753165463 +0000 2024-03-16T20:44:07.436 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-03-16T20:44:07.436 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-16T20:44:07.490 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-03-16T20:44:07.490 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-03-16T20:44:07.490 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.00030282 s, 1.7 MB/s 2024-03-16T20:44:07.492 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-16T20:44:07.540 DEBUG:teuthology.orchestra.run.smithi195:> stat /dev/vg_nvme/lv_4 2024-03-16T20:44:07.590 INFO:teuthology.orchestra.run.smithi195.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-16T20:44:07.590 INFO:teuthology.orchestra.run.smithi195.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-16T20:44:07.590 INFO:teuthology.orchestra.run.smithi195.stdout:Device: 5h/5d Inode: 935 Links: 1 2024-03-16T20:44:07.591 INFO:teuthology.orchestra.run.smithi195.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-16T20:44:07.591 INFO:teuthology.orchestra.run.smithi195.stdout:Access: 2024-03-16 20:38:00.521158612 +0000 2024-03-16T20:44:07.591 INFO:teuthology.orchestra.run.smithi195.stdout:Modify: 2024-03-16 20:38:00.241161109 +0000 2024-03-16T20:44:07.591 INFO:teuthology.orchestra.run.smithi195.stdout:Change: 2024-03-16 20:38:00.241161109 +0000 2024-03-16T20:44:07.591 INFO:teuthology.orchestra.run.smithi195.stdout: Birth: - 2024-03-16T20:44:07.591 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-16T20:44:07.650 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records in 2024-03-16T20:44:07.650 INFO:teuthology.orchestra.run.smithi195.stderr:1+0 records out 2024-03-16T20:44:07.650 INFO:teuthology.orchestra.run.smithi195.stderr:512 bytes copied, 0.000303228 s, 1.7 MB/s 2024-03-16T20:44:07.651 DEBUG:teuthology.orchestra.run.smithi195:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-16T20:44:07.705 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-16T20:44:07.705 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-16T20:44:07.705 INFO:tasks.ceph:Generating config... 2024-03-16T20:44:07.706 INFO:tasks.ceph:[client] client mount timeout = 600 2024-03-16T20:44:07.706 INFO:tasks.ceph:[client] debug client = 20 2024-03-16T20:44:07.706 INFO:tasks.ceph:[client] debug ms = 1 2024-03-16T20:44:07.706 INFO:tasks.ceph:[client] pid file = 2024-03-16T20:44:07.707 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-03-16T20:44:07.707 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-03-16T20:44:07.707 INFO:tasks.ceph:[mds] debug mds = 20 2024-03-16T20:44:07.707 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-03-16T20:44:07.707 INFO:tasks.ceph:[mds] debug ms = 1 2024-03-16T20:44:07.707 INFO:tasks.ceph:[mds] mds debug frag = True 2024-03-16T20:44:07.707 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-03-16T20:44:07.707 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-03-16T20:44:07.707 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-03-16T20:44:07.707 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-03-16T20:44:07.707 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-03-16T20:44:07.707 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-03-16T20:44:07.707 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-03-16T20:44:07.707 INFO:tasks.ceph:[mgr] debug ms = 1 2024-03-16T20:44:07.707 INFO:tasks.ceph:[mon] debug mon = 20 2024-03-16T20:44:07.708 INFO:tasks.ceph:[mon] debug ms = 1 2024-03-16T20:44:07.708 INFO:tasks.ceph:[mon] debug paxos = 20 2024-03-16T20:44:07.708 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-03-16T20:44:07.708 INFO:tasks.ceph:[osd] bdev async discard = True 2024-03-16T20:44:07.708 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-03-16T20:44:07.708 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-03-16T20:44:07.708 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-03-16T20:44:07.708 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-03-16T20:44:07.708 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-03-16T20:44:07.708 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-03-16T20:44:07.708 INFO:tasks.ceph:[osd] debug ms = 1 2024-03-16T20:44:07.708 INFO:tasks.ceph:[osd] debug osd = 20 2024-03-16T20:44:07.708 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-03-16T20:44:07.708 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-03-16T20:44:07.708 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-03-16T20:44:07.709 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-03-16T20:44:07.709 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-03-16T20:44:07.709 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-03-16T20:44:07.709 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-03-16T20:44:07.709 INFO:tasks.ceph:Setting up mon.a... 2024-03-16T20:44:07.709 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-03-16T20:44:07.741 INFO:teuthology.orchestra.run.smithi122.stdout:creating /etc/ceph/ceph.keyring 2024-03-16T20:44:07.749 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-03-16T20:44:07.824 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-16T20:44:07.879 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.122'), ('mon.b', '172.21.15.195'), ('mon.c', '[v2:172.21.15.195:3301,v1:172.21.15.195:6790]')] 2024-03-16T20:44:07.880 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.122,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.b': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-03-16T20:44:07.880 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:07.880 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-03-16T20:44:07.924 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.122 --add b 172.21.15.195 --addv c '[v2:172.21.15.195:3301,v1:172.21.15.195:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-03-16T20:44:08.010 INFO:teuthology.orchestra.run.smithi122.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-03-16T20:44:08.010 INFO:teuthology.orchestra.run.smithi122.stdout:monmaptool: generated fsid ea4c964c-3481-413d-a556-a3fc2950fdd6 2024-03-16T20:44:08.010 INFO:teuthology.orchestra.run.smithi122.stdout:setting min_mon_release = quincy 2024-03-16T20:44:08.010 INFO:teuthology.orchestra.run.smithi122.stdout:epoch 0 2024-03-16T20:44:08.010 INFO:teuthology.orchestra.run.smithi122.stdout:fsid ea4c964c-3481-413d-a556-a3fc2950fdd6 2024-03-16T20:44:08.010 INFO:teuthology.orchestra.run.smithi122.stdout:last_changed 2024-03-16T20:44:08.010728+0000 2024-03-16T20:44:08.010 INFO:teuthology.orchestra.run.smithi122.stdout:created 2024-03-16T20:44:08.010728+0000 2024-03-16T20:44:08.010 INFO:teuthology.orchestra.run.smithi122.stdout:min_mon_release 17 (quincy) 2024-03-16T20:44:08.011 INFO:teuthology.orchestra.run.smithi122.stdout:election_strategy: 1 2024-03-16T20:44:08.011 INFO:teuthology.orchestra.run.smithi122.stdout:0: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.a 2024-03-16T20:44:08.011 INFO:teuthology.orchestra.run.smithi122.stdout:1: [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] mon.b 2024-03-16T20:44:08.011 INFO:teuthology.orchestra.run.smithi122.stdout:2: [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] mon.c 2024-03-16T20:44:08.011 INFO:teuthology.orchestra.run.smithi122.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-03-16T20:44:08.012 DEBUG:teuthology.orchestra.run.smithi122:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-03-16T20:44:08.060 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ea4c964c-3481-413d-a556-a3fc2950fdd6... 2024-03-16T20:44:08.061 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-16T20:44:08.105 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-03-16T20:44:08.133 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2024-03-16T20:44:08.133 INFO:teuthology.orchestra.run.smithi122.stdout: chdir = "" 2024-03-16T20:44:08.133 INFO:teuthology.orchestra.run.smithi122.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-16T20:44:08.134 INFO:teuthology.orchestra.run.smithi122.stdout: auth supported = cephx 2024-03-16T20:44:08.134 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.134 INFO:teuthology.orchestra.run.smithi122.stdout: filestore xattr use omap = true 2024-03-16T20:44:08.134 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.134 INFO:teuthology.orchestra.run.smithi122.stdout: mon clock drift allowed = 1.000 2024-03-16T20:44:08.134 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.134 INFO:teuthology.orchestra.run.smithi122.stdout: osd crush chooseleaf type = 0 2024-03-16T20:44:08.134 INFO:teuthology.orchestra.run.smithi122.stdout: auth debug = true 2024-03-16T20:44:08.134 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.134 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on old message = true 2024-03-16T20:44:08.134 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on bug = true 2024-03-16T20:44:08.134 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.135 INFO:teuthology.orchestra.run.smithi122.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-16T20:44:08.135 INFO:teuthology.orchestra.run.smithi122.stdout: mon pg warn max object skew = 0 2024-03-16T20:44:08.135 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.135 INFO:teuthology.orchestra.run.smithi122.stdout: # disable pg_autoscaler by default for new pools 2024-03-16T20:44:08.135 INFO:teuthology.orchestra.run.smithi122.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-16T20:44:08.135 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.135 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default size = 2 2024-03-16T20:44:08.135 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.135 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow primary affinity = true 2024-03-16T20:44:08.135 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow pg remap = true 2024-03-16T20:44:08.135 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on legacy crush tunables = false 2024-03-16T20:44:08.135 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on crush straw calc version zero = false 2024-03-16T20:44:08.135 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on no sortbitwise = false 2024-03-16T20:44:08.135 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on osd down out interval zero = false 2024-03-16T20:44:08.135 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on too few osds = false 2024-03-16T20:44:08.136 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-16T20:44:08.136 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-16T20:44:08.136 INFO:teuthology.orchestra.run.smithi122.stdout: mon_allow_pool_size_one = true 2024-03-16T20:44:08.136 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.136 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-16T20:44:08.136 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.136 INFO:teuthology.orchestra.run.smithi122.stdout: osd default data pool replay window = 5 2024-03-16T20:44:08.136 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.136 INFO:teuthology.orchestra.run.smithi122.stdout: mon allow pool delete = true 2024-03-16T20:44:08.136 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.136 INFO:teuthology.orchestra.run.smithi122.stdout: mon cluster log file level = debug 2024-03-16T20:44:08.136 INFO:teuthology.orchestra.run.smithi122.stdout: debug asserts on shutdown = true 2024-03-16T20:44:08.136 INFO:teuthology.orchestra.run.smithi122.stdout: mon health detail to clog = false 2024-03-16T20:44:08.136 INFO:teuthology.orchestra.run.smithi122.stdout: mon host = "172.21.15.122,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2024-03-16T20:44:08.136 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = ea4c964c-3481-413d-a556-a3fc2950fdd6 2024-03-16T20:44:08.137 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.137 INFO:teuthology.orchestra.run.smithi122.stdout:[osd] 2024-03-16T20:44:08.137 INFO:teuthology.orchestra.run.smithi122.stdout: osd journal size = 100 2024-03-16T20:44:08.137 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.137 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub load threshold = 5.0 2024-03-16T20:44:08.137 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub max interval = 600 2024-03-16T20:44:08.137 INFO:teuthology.orchestra.run.smithi122.stdout: osd mclock profile = high_recovery_ops 2024-03-16T20:44:08.137 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.137 INFO:teuthology.orchestra.run.smithi122.stdout: osd recover clone overlap = true 2024-03-16T20:44:08.137 INFO:teuthology.orchestra.run.smithi122.stdout: osd recovery max chunk = 1048576 2024-03-16T20:44:08.137 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.137 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug shutdown = true 2024-03-16T20:44:08.137 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug op order = true 2024-03-16T20:44:08.137 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify stray on activate = true 2024-03-16T20:44:08.137 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug trim objects = true 2024-03-16T20:44:08.138 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.138 INFO:teuthology.orchestra.run.smithi122.stdout: osd open classes on start = true 2024-03-16T20:44:08.138 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug pg log writeout = true 2024-03-16T20:44:08.138 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.138 INFO:teuthology.orchestra.run.smithi122.stdout: osd deep scrub update digest min age = 30 2024-03-16T20:44:08.138 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.138 INFO:teuthology.orchestra.run.smithi122.stdout: osd map max advance = 10 2024-03-16T20:44:08.138 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.138 INFO:teuthology.orchestra.run.smithi122.stdout: journal zero on create = true 2024-03-16T20:44:08.138 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.138 INFO:teuthology.orchestra.run.smithi122.stdout: filestore ondisk finisher threads = 3 2024-03-16T20:44:08.138 INFO:teuthology.orchestra.run.smithi122.stdout: filestore apply finisher threads = 3 2024-03-16T20:44:08.138 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.138 INFO:teuthology.orchestra.run.smithi122.stdout: bdev debug aio = true 2024-03-16T20:44:08.139 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug misdirected ops = true 2024-03-16T20:44:08.139 INFO:teuthology.orchestra.run.smithi122.stdout: bdev async discard = True 2024-03-16T20:44:08.139 INFO:teuthology.orchestra.run.smithi122.stdout: bdev enable discard = True 2024-03-16T20:44:08.139 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore allocator = bitmap 2024-03-16T20:44:08.139 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore block size = 96636764160 2024-03-16T20:44:08.139 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore fsck on mount = True 2024-03-16T20:44:08.139 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluefs = 1/20 2024-03-16T20:44:08.139 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluestore = 1/20 2024-03-16T20:44:08.139 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-03-16T20:44:08.139 INFO:teuthology.orchestra.run.smithi122.stdout: debug osd = 20 2024-03-16T20:44:08.139 INFO:teuthology.orchestra.run.smithi122.stdout: debug rocksdb = 4/10 2024-03-16T20:44:08.139 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd backfillfull_ratio = 0.85 2024-03-16T20:44:08.139 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd full ratio = 0.9 2024-03-16T20:44:08.139 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd nearfull ratio = 0.8 2024-03-16T20:44:08.140 INFO:teuthology.orchestra.run.smithi122.stdout: osd failsafe full ratio = 0.95 2024-03-16T20:44:08.140 INFO:teuthology.orchestra.run.smithi122.stdout: osd objectstore = bluestore 2024-03-16T20:44:08.140 INFO:teuthology.orchestra.run.smithi122.stdout: osd op complaint time = 180 2024-03-16T20:44:08.140 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.140 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr] 2024-03-16T20:44:08.140 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-03-16T20:44:08.140 INFO:teuthology.orchestra.run.smithi122.stdout: debug mgr = 20 2024-03-16T20:44:08.140 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-03-16T20:44:08.140 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-03-16T20:44:08.140 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-03-16T20:44:08.140 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-03-16T20:44:08.140 INFO:teuthology.orchestra.run.smithi122.stdout: mgr/telemetry/nag = false 2024-03-16T20:44:08.140 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.140 INFO:teuthology.orchestra.run.smithi122.stdout:[mon] 2024-03-16T20:44:08.140 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-03-16T20:44:08.141 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-03-16T20:44:08.141 INFO:teuthology.orchestra.run.smithi122.stdout: debug paxos = 20 2024-03-16T20:44:08.141 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-03-16T20:44:08.141 INFO:teuthology.orchestra.run.smithi122.stdout: mon data avail warn = 5 2024-03-16T20:44:08.141 INFO:teuthology.orchestra.run.smithi122.stdout: mon mgr mkfs grace = 240 2024-03-16T20:44:08.141 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-03-16T20:44:08.141 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd reporter subtree level = osd 2024-03-16T20:44:08.141 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd prime pg temp = true 2024-03-16T20:44:08.141 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-03-16T20:44:08.141 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.141 INFO:teuthology.orchestra.run.smithi122.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-16T20:44:08.141 INFO:teuthology.orchestra.run.smithi122.stdout: auth mon ticket ttl = 660 # 11m 2024-03-16T20:44:08.141 INFO:teuthology.orchestra.run.smithi122.stdout: auth service ticket ttl = 240 # 4m 2024-03-16T20:44:08.141 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.141 INFO:teuthology.orchestra.run.smithi122.stdout: # don't complain about insecure global_id in the test suite 2024-03-16T20:44:08.142 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-16T20:44:08.142 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-16T20:44:08.142 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.142 INFO:teuthology.orchestra.run.smithi122.stdout: # 1m isn't quite enough 2024-03-16T20:44:08.142 INFO:teuthology.orchestra.run.smithi122.stdout: mon_down_mkfs_grace = 2m 2024-03-16T20:44:08.142 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.142 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_filestore_osds = false 2024-03-16T20:44:08.142 INFO:teuthology.orchestra.run.smithi122.stdout: mon op complaint time = 120 2024-03-16T20:44:08.142 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:08.142 INFO:teuthology.orchestra.run.smithi122.stdout:[client] 2024-03-16T20:44:08.142 INFO:teuthology.orchestra.run.smithi122.stdout: rgw cache enabled = true 2024-03-16T20:44:08.142 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable ops log = true 2024-03-16T20:44:08.142 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable usage log = true 2024-03-16T20:44:08.142 INFO:teuthology.orchestra.run.smithi122.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-16T20:44:08.143 INFO:teuthology.orchestra.run.smithi122.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-16T20:44:08.143 INFO:teuthology.orchestra.run.smithi122.stdout: client mount timeout = 600 2024-03-16T20:44:08.143 INFO:teuthology.orchestra.run.smithi122.stdout: debug client = 20 2024-03-16T20:44:08.143 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-03-16T20:44:08.143 INFO:teuthology.orchestra.run.smithi122.stdout: pid file = "" 2024-03-16T20:44:08.143 INFO:teuthology.orchestra.run.smithi122.stdout: rados mon op timeout = 900 2024-03-16T20:44:08.143 INFO:teuthology.orchestra.run.smithi122.stdout: rados osd op timeout = 900 2024-03-16T20:44:08.143 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.a] 2024-03-16T20:44:08.143 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.b] 2024-03-16T20:44:08.143 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.c] 2024-03-16T20:44:08.143 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.a] 2024-03-16T20:44:08.143 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.b] 2024-03-16T20:44:08.143 INFO:teuthology.orchestra.run.smithi122.stdout:[mds] 2024-03-16T20:44:08.143 INFO:teuthology.orchestra.run.smithi122.stdout: debug mds = 20 2024-03-16T20:44:08.143 INFO:teuthology.orchestra.run.smithi122.stdout: debug mds balancer = 20 2024-03-16T20:44:08.143 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-03-16T20:44:08.144 INFO:teuthology.orchestra.run.smithi122.stdout: mds debug frag = True 2024-03-16T20:44:08.144 INFO:teuthology.orchestra.run.smithi122.stdout: mds debug scatterstat = True 2024-03-16T20:44:08.144 INFO:teuthology.orchestra.run.smithi122.stdout: mds op complaint time = 180 2024-03-16T20:44:08.144 INFO:teuthology.orchestra.run.smithi122.stdout: mds verify scatter = True 2024-03-16T20:44:08.144 INFO:teuthology.orchestra.run.smithi122.stdout: osd op complaint time = 180 2024-03-16T20:44:08.144 INFO:teuthology.orchestra.run.smithi122.stdout: rados mon op timeout = 900 2024-03-16T20:44:08.144 INFO:teuthology.orchestra.run.smithi122.stdout: rados osd op timeout = 900 2024-03-16T20:44:08.144 INFO:teuthology.orchestra.run.smithi195.stdout:[global] 2024-03-16T20:44:08.144 INFO:teuthology.orchestra.run.smithi195.stdout: chdir = "" 2024-03-16T20:44:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-03-16T20:44:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: auth supported = cephx 2024-03-16T20:44:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: filestore xattr use omap = true 2024-03-16T20:44:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: mon clock drift allowed = 1.000 2024-03-16T20:44:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: osd crush chooseleaf type = 0 2024-03-16T20:44:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: auth debug = true 2024-03-16T20:44:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on old message = true 2024-03-16T20:44:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: ms die on bug = true 2024-03-16T20:44:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.145 INFO:teuthology.orchestra.run.smithi195.stdout: mon max pg per osd = 10000 # >= luminous 2024-03-16T20:44:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: mon pg warn max object skew = 0 2024-03-16T20:44:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: # disable pg_autoscaler by default for new pools 2024-03-16T20:44:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: osd_pool_default_pg_autoscale_mode = off 2024-03-16T20:44:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default size = 2 2024-03-16T20:44:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow primary affinity = true 2024-03-16T20:44:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd allow pg remap = true 2024-03-16T20:44:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on legacy crush tunables = false 2024-03-16T20:44:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on crush straw calc version zero = false 2024-03-16T20:44:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on no sortbitwise = false 2024-03-16T20:44:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on osd down out interval zero = false 2024-03-16T20:44:08.146 INFO:teuthology.orchestra.run.smithi195.stdout: mon warn on too few osds = false 2024-03-16T20:44:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-03-16T20:44:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_pool_no_redundancy = false 2024-03-16T20:44:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: mon_allow_pool_size_one = true 2024-03-16T20:44:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-03-16T20:44:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: osd default data pool replay window = 5 2024-03-16T20:44:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: mon allow pool delete = true 2024-03-16T20:44:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: mon cluster log file level = debug 2024-03-16T20:44:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: debug asserts on shutdown = true 2024-03-16T20:44:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: mon health detail to clog = false 2024-03-16T20:44:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: mon host = "172.21.15.122,172.21.15.195,[v2:172.21.15.195:3301,v1:172.21.15.195:6790]" 2024-03-16T20:44:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: fsid = ea4c964c-3481-413d-a556-a3fc2950fdd6 2024-03-16T20:44:08.147 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.148 INFO:teuthology.orchestra.run.smithi195.stdout:[osd] 2024-03-16T20:44:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: osd journal size = 100 2024-03-16T20:44:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub load threshold = 5.0 2024-03-16T20:44:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: osd scrub max interval = 600 2024-03-16T20:44:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: osd mclock profile = high_recovery_ops 2024-03-16T20:44:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: osd recover clone overlap = true 2024-03-16T20:44:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: osd recovery max chunk = 1048576 2024-03-16T20:44:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug shutdown = true 2024-03-16T20:44:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug op order = true 2024-03-16T20:44:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug verify stray on activate = true 2024-03-16T20:44:08.148 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug trim objects = true 2024-03-16T20:44:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: osd open classes on start = true 2024-03-16T20:44:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug pg log writeout = true 2024-03-16T20:44:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: osd deep scrub update digest min age = 30 2024-03-16T20:44:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: osd map max advance = 10 2024-03-16T20:44:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: journal zero on create = true 2024-03-16T20:44:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: filestore ondisk finisher threads = 3 2024-03-16T20:44:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: filestore apply finisher threads = 3 2024-03-16T20:44:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: bdev debug aio = true 2024-03-16T20:44:08.149 INFO:teuthology.orchestra.run.smithi195.stdout: osd debug misdirected ops = true 2024-03-16T20:44:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: bdev async discard = True 2024-03-16T20:44:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: bdev enable discard = True 2024-03-16T20:44:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore allocator = bitmap 2024-03-16T20:44:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore block size = 96636764160 2024-03-16T20:44:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: bluestore fsck on mount = True 2024-03-16T20:44:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluefs = 1/20 2024-03-16T20:44:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: debug bluestore = 1/20 2024-03-16T20:44:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-03-16T20:44:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: debug osd = 20 2024-03-16T20:44:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: debug rocksdb = 4/10 2024-03-16T20:44:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd backfillfull_ratio = 0.85 2024-03-16T20:44:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd full ratio = 0.9 2024-03-16T20:44:08.150 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd nearfull ratio = 0.8 2024-03-16T20:44:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: osd failsafe full ratio = 0.95 2024-03-16T20:44:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: osd objectstore = bluestore 2024-03-16T20:44:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2024-03-16T20:44:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.151 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr] 2024-03-16T20:44:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-03-16T20:44:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: debug mgr = 20 2024-03-16T20:44:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-03-16T20:44:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-03-16T20:44:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-03-16T20:44:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-03-16T20:44:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: mgr/telemetry/nag = false 2024-03-16T20:44:08.151 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.151 INFO:teuthology.orchestra.run.smithi195.stdout:[mon] 2024-03-16T20:44:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-03-16T20:44:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: debug mon = 20 2024-03-16T20:44:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: debug paxos = 20 2024-03-16T20:44:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: debug auth = 20 2024-03-16T20:44:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: mon data avail warn = 5 2024-03-16T20:44:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: mon mgr mkfs grace = 240 2024-03-16T20:44:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min pgs per osd = 4 2024-03-16T20:44:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd reporter subtree level = osd 2024-03-16T20:44:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: mon osd prime pg temp = true 2024-03-16T20:44:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: mon reweight min bytes per osd = 10 2024-03-16T20:44:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-03-16T20:44:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: auth mon ticket ttl = 660 # 11m 2024-03-16T20:44:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: auth service ticket ttl = 240 # 4m 2024-03-16T20:44:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.152 INFO:teuthology.orchestra.run.smithi195.stdout: # don't complain about insecure global_id in the test suite 2024-03-16T20:44:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-03-16T20:44:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-03-16T20:44:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: # 1m isn't quite enough 2024-03-16T20:44:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: mon_down_mkfs_grace = 2m 2024-03-16T20:44:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: mon_warn_on_filestore_osds = false 2024-03-16T20:44:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: mon op complaint time = 120 2024-03-16T20:44:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: 2024-03-16T20:44:08.153 INFO:teuthology.orchestra.run.smithi195.stdout:[client] 2024-03-16T20:44:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: rgw cache enabled = true 2024-03-16T20:44:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable ops log = true 2024-03-16T20:44:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: rgw enable usage log = true 2024-03-16T20:44:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-03-16T20:44:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-03-16T20:44:08.153 INFO:teuthology.orchestra.run.smithi195.stdout: client mount timeout = 600 2024-03-16T20:44:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: debug client = 20 2024-03-16T20:44:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-03-16T20:44:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: pid file = "" 2024-03-16T20:44:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2024-03-16T20:44:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2024-03-16T20:44:08.154 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.a] 2024-03-16T20:44:08.154 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.b] 2024-03-16T20:44:08.154 INFO:teuthology.orchestra.run.smithi195.stdout:[mon.c] 2024-03-16T20:44:08.154 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2024-03-16T20:44:08.154 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2024-03-16T20:44:08.154 INFO:teuthology.orchestra.run.smithi195.stdout:[mds] 2024-03-16T20:44:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds = 20 2024-03-16T20:44:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: debug mds balancer = 20 2024-03-16T20:44:08.154 INFO:teuthology.orchestra.run.smithi195.stdout: debug ms = 1 2024-03-16T20:44:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug frag = True 2024-03-16T20:44:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: mds debug scatterstat = True 2024-03-16T20:44:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: mds op complaint time = 180 2024-03-16T20:44:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: mds verify scatter = True 2024-03-16T20:44:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: osd op complaint time = 180 2024-03-16T20:44:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: rados mon op timeout = 900 2024-03-16T20:44:08.155 INFO:teuthology.orchestra.run.smithi195.stdout: rados osd op timeout = 900 2024-03-16T20:44:08.156 INFO:tasks.ceph:Creating admin key on mon.a... 2024-03-16T20:44:08.156 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-03-16T20:44:08.247 INFO:tasks.ceph:Copying monmap to all nodes... 2024-03-16T20:44:08.247 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:08.247 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-03-16T20:44:08.252 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:08.252 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-03-16T20:44:08.300 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi122.front.sepia.ceph.com 2024-03-16T20:44:08.301 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:08.301 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-16T20:44:08.301 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-16T20:44:08.365 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:08.366 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-16T20:44:08.412 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi195.front.sepia.ceph.com 2024-03-16T20:44:08.413 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:08.413 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd of=/etc/ceph/ceph.keyring 2024-03-16T20:44:08.413 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-03-16T20:44:08.435 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:08.435 DEBUG:teuthology.orchestra.run.smithi195:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-03-16T20:44:08.488 INFO:tasks.ceph:Setting up mon nodes... 2024-03-16T20:44:08.488 INFO:tasks.ceph:Setting up mgr nodes... 2024-03-16T20:44:08.488 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-03-16T20:44:08.532 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-03-16T20:44:08.535 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-03-16T20:44:08.579 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-03-16T20:44:08.612 INFO:tasks.ceph:Setting up mds nodes... 2024-03-16T20:44:08.613 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-03-16T20:44:08.656 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-03-16T20:44:08.659 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-03-16T20:44:08.712 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-03-16T20:44:08.756 INFO:teuthology.orchestra.run.smithi195.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-03-16T20:44:08.759 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-03-16T20:44:08.822 INFO:tasks.ceph_client:Setting up client nodes... 2024-03-16T20:44:08.822 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-16T20:44:08.857 INFO:teuthology.orchestra.run.smithi122.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-03-16T20:44:08.869 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-03-16T20:44:08.869 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi195.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-03-16T20:44:08.869 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-03-16T20:44:08.924 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-16T20:44:08.924 INFO:tasks.ceph:role: osd.0 2024-03-16T20:44:08.924 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com 2024-03-16T20:44:08.924 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-16T20:44:08.988 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T20:44:08.989 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T20:44:08.989 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T20:44:08.989 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-16T20:44:08.989 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T20:44:08.989 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-03-16T20:44:08.989 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T20:44:08.989 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-16T20:44:08.989 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T20:44:08.989 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T20:44:09.433 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-03-16T20:44:09.435 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-03-16T20:44:09.435 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-03-16T20:44:09.531 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-03-16T20:44:09.540 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-03-16T20:44:09.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T20:44:09.541 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-03-16T20:44:09.594 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-16T20:44:09.594 INFO:tasks.ceph:role: osd.1 2024-03-16T20:44:09.594 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com 2024-03-16T20:44:09.594 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-16T20:44:09.655 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T20:44:09.655 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T20:44:09.655 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T20:44:09.655 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-16T20:44:09.655 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T20:44:09.655 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-03-16T20:44:09.655 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T20:44:09.655 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-16T20:44:09.655 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T20:44:09.655 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T20:44:10.121 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-03-16T20:44:10.123 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-03-16T20:44:10.124 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-03-16T20:44:10.268 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-03-16T20:44:10.279 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-03-16T20:44:10.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T20:44:10.280 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-03-16T20:44:10.336 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-16T20:44:10.337 INFO:tasks.ceph:role: osd.2 2024-03-16T20:44:10.337 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com 2024-03-16T20:44:10.337 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-16T20:44:10.400 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T20:44:10.400 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T20:44:10.401 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T20:44:10.401 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-16T20:44:10.401 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T20:44:10.401 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-03-16T20:44:10.401 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T20:44:10.401 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-16T20:44:10.401 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T20:44:10.401 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T20:44:10.876 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-03-16T20:44:10.879 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-03-16T20:44:10.879 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-03-16T20:44:11.023 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-03-16T20:44:11.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T20:44:11.035 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-03-16T20:44:11.035 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-03-16T20:44:11.088 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-03-16T20:44:11.088 INFO:tasks.ceph:role: osd.3 2024-03-16T20:44:11.088 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi122.front.sepia.ceph.com 2024-03-16T20:44:11.088 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-16T20:44:11.152 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T20:44:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T20:44:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T20:44:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-16T20:44:11.152 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T20:44:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-03-16T20:44:11.152 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T20:44:11.152 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-16T20:44:11.152 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T20:44:11.152 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T20:44:11.574 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-03-16T20:44:11.575 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-03-16T20:44:11.576 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-03-16T20:44:11.669 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-03-16T20:44:11.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T20:44:11.679 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-03-16T20:44:11.680 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T20:44:11.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:11.791+0000 7f45d07a37c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-03-16T20:44:11.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:11.795+0000 7f45d07a37c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-03-16T20:44:11.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:11.795+0000 7f45d07a37c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-16T20:44:11.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:11.795+0000 7f45d07a37c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-16T20:44:11.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:11.795+0000 7f45d07a37c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-03-16T20:44:11.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:11.799+0000 7f45d07a37c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-03-16T20:44:14.796 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-03-16T20:44:14.810 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T20:44:14.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:14.887+0000 7f88abead7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-03-16T20:44:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:14.887+0000 7f88abead7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-03-16T20:44:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:14.887+0000 7f88abead7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-16T20:44:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:14.887+0000 7f88abead7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-16T20:44:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:14.887+0000 7f88abead7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-03-16T20:44:14.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:14.887+0000 7f88abead7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-03-16T20:44:17.856 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-03-16T20:44:17.912 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T20:44:17.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:17.991+0000 7f80c80647c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-03-16T20:44:17.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:17.991+0000 7f80c80647c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-03-16T20:44:17.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:17.991+0000 7f80c80647c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-16T20:44:17.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:17.991+0000 7f80c80647c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-16T20:44:17.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:17.991+0000 7f80c80647c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-03-16T20:44:17.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:17.991+0000 7f80c80647c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-03-16T20:44:21.020 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-03-16T20:44:21.076 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T20:44:21.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:21.155+0000 7fde7c2107c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-03-16T20:44:21.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:21.155+0000 7fde7c2107c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-03-16T20:44:21.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:21.155+0000 7fde7c2107c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-16T20:44:21.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:21.155+0000 7fde7c2107c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-16T20:44:21.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:21.155+0000 7fde7c2107c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-03-16T20:44:21.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:21.155+0000 7fde7c2107c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-03-16T20:44:24.130 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-03-16T20:44:24.185 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-03-16T20:44:24.200 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-16T20:44:24.200 INFO:tasks.ceph:role: osd.4 2024-03-16T20:44:24.200 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com 2024-03-16T20:44:24.200 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-03-16T20:44:24.256 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T20:44:24.256 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T20:44:24.256 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T20:44:24.256 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-16T20:44:24.256 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T20:44:24.256 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-03-16T20:44:24.256 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T20:44:24.257 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-16T20:44:24.257 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T20:44:24.257 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T20:44:24.280 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-03-16T20:44:24.282 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-03-16T20:44:24.283 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-03-16T20:44:24.340 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-03-16T20:44:24.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T20:44:24.391 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-03-16T20:44:24.392 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-03-16T20:44:24.444 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-16T20:44:24.444 INFO:tasks.ceph:role: osd.5 2024-03-16T20:44:24.445 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com 2024-03-16T20:44:24.445 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-03-16T20:44:24.496 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T20:44:24.496 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T20:44:24.496 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T20:44:24.496 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-16T20:44:24.496 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T20:44:24.496 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-03-16T20:44:24.496 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T20:44:24.497 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-16T20:44:24.497 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T20:44:24.497 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T20:44:24.520 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-03-16T20:44:24.521 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-03-16T20:44:24.521 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-03-16T20:44:24.579 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-03-16T20:44:24.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T20:44:24.631 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-03-16T20:44:24.632 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-03-16T20:44:24.684 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-16T20:44:24.684 INFO:tasks.ceph:role: osd.6 2024-03-16T20:44:24.684 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com 2024-03-16T20:44:24.684 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-03-16T20:44:24.739 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T20:44:24.739 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T20:44:24.739 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T20:44:24.739 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-16T20:44:24.739 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T20:44:24.739 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-03-16T20:44:24.739 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T20:44:24.739 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-16T20:44:24.739 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T20:44:24.739 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T20:44:24.763 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-03-16T20:44:24.765 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-03-16T20:44:24.765 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-03-16T20:44:24.827 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-03-16T20:44:24.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T20:44:24.880 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-03-16T20:44:24.880 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-03-16T20:44:24.934 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-03-16T20:44:24.934 INFO:tasks.ceph:role: osd.7 2024-03-16T20:44:24.934 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com 2024-03-16T20:44:24.935 DEBUG:teuthology.orchestra.run.smithi195:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-03-16T20:44:24.991 INFO:teuthology.orchestra.run.smithi195.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-03-16T20:44:24.991 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 attr=2, projid32bit=1 2024-03-16T20:44:24.991 INFO:teuthology.orchestra.run.smithi195.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-03-16T20:44:24.991 INFO:teuthology.orchestra.run.smithi195.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-03-16T20:44:24.991 INFO:teuthology.orchestra.run.smithi195.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-03-16T20:44:24.991 INFO:teuthology.orchestra.run.smithi195.stdout: = sunit=0 swidth=0 blks 2024-03-16T20:44:24.991 INFO:teuthology.orchestra.run.smithi195.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-03-16T20:44:24.992 INFO:teuthology.orchestra.run.smithi195.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-03-16T20:44:24.992 INFO:teuthology.orchestra.run.smithi195.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-03-16T20:44:24.992 INFO:teuthology.orchestra.run.smithi195.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-03-16T20:44:25.014 INFO:teuthology.orchestra.run.smithi195.stdout:Discarding blocks...Done. 2024-03-16T20:44:25.016 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi195.front.sepia.ceph.com -o noatime 2024-03-16T20:44:25.016 DEBUG:teuthology.orchestra.run.smithi195:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-03-16T20:44:25.076 DEBUG:teuthology.orchestra.run.smithi195:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-03-16T20:44:25.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T20:44:25.128 INFO:teuthology.orchestra.run.smithi195.stderr:sudo: /sbin/restorecon: command not found 2024-03-16T20:44:25.128 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T20:44:25.236 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:25.230+0000 7f7fc59a97c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-03-16T20:44:25.236 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:25.230+0000 7f7fc59a97c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-03-16T20:44:25.236 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:25.230+0000 7f7fc59a97c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-16T20:44:25.236 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:25.230+0000 7f7fc59a97c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-16T20:44:25.236 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:25.230+0000 7f7fc59a97c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-03-16T20:44:25.237 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:25.230+0000 7f7fc59a97c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-03-16T20:44:28.210 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-03-16T20:44:28.224 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T20:44:28.305 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:28.298+0000 7f1bb65f97c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-03-16T20:44:28.305 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:28.298+0000 7f1bb65f97c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-03-16T20:44:28.305 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:28.298+0000 7f1bb65f97c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-16T20:44:28.305 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:28.298+0000 7f1bb65f97c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-16T20:44:28.305 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:28.298+0000 7f1bb65f97c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-03-16T20:44:28.305 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:28.298+0000 7f1bb65f97c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-03-16T20:44:31.265 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-03-16T20:44:31.323 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T20:44:31.406 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:31.402+0000 7f557df6c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-03-16T20:44:31.406 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:31.402+0000 7f557df6c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-03-16T20:44:31.406 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:31.402+0000 7f557df6c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-16T20:44:31.406 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:31.402+0000 7f557df6c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-16T20:44:31.406 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:31.402+0000 7f557df6c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-03-16T20:44:31.406 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:31.402+0000 7f557df6c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-03-16T20:44:34.366 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-03-16T20:44:34.424 DEBUG:teuthology.orchestra.run.smithi195:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-03-16T20:44:34.504 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:34.502+0000 7fcea2a117c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-03-16T20:44:34.504 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:34.502+0000 7fcea2a117c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-03-16T20:44:34.504 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:34.502+0000 7fcea2a117c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-16T20:44:34.504 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:34.502+0000 7fcea2a117c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-16T20:44:34.504 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:34.502+0000 7fcea2a117c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-03-16T20:44:34.505 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:34.502+0000 7fcea2a117c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-03-16T20:44:36.361 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T20:44:36.363 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T20:44:37.473 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-03-16T20:44:37.485 INFO:tasks.ceph:Reading keys from all nodes... 2024-03-16T20:44:37.486 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:37.486 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-03-16T20:44:37.500 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:37.500 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-03-16T20:44:37.555 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:37.555 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-03-16T20:44:37.611 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:37.611 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-03-16T20:44:37.663 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:37.663 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-03-16T20:44:37.719 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:37.719 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-03-16T20:44:37.776 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:37.776 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-03-16T20:44:37.789 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:37.790 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-03-16T20:44:37.843 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:37.843 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-03-16T20:44:37.898 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:37.898 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-03-16T20:44:37.950 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:37.950 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-03-16T20:44:38.002 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:38.002 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-03-16T20:44:38.058 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:38.058 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-03-16T20:44:38.064 INFO:tasks.ceph:Adding keys to all mons... 2024-03-16T20:44:38.064 DEBUG:teuthology.orchestra.run.smithi122:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-16T20:44:38.109 DEBUG:teuthology.orchestra.run.smithi195:> sudo tee -a /etc/ceph/ceph.keyring 2024-03-16T20:44:38.118 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr.y] 2024-03-16T20:44:38.118 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCYBPZlvgfHHxAAeorpgzjZnG6znj3MHGlTvA== 2024-03-16T20:44:38.118 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.a] 2024-03-16T20:44:38.118 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCYBPZldiQwJxAAqp05o6yqMhy6pZoNHnDHKA== 2024-03-16T20:44:38.118 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.0] 2024-03-16T20:44:38.118 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCbBPZlS35pLxAAh2wHT8Aeinet1hO5A1ioyg== 2024-03-16T20:44:38.118 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.1] 2024-03-16T20:44:38.119 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCeBPZl+cEDNRAAPCKGGs2VdPWHsPvgdOnxIQ== 2024-03-16T20:44:38.119 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.2] 2024-03-16T20:44:38.119 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQChBPZlYcdGOxAAW+DGd7sDp76oQ7YgETtrBg== 2024-03-16T20:44:38.119 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.3] 2024-03-16T20:44:38.119 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQClBPZleeJnCRAAueEygVrApZN0Cc6L2q2kng== 2024-03-16T20:44:38.119 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr.x] 2024-03-16T20:44:38.119 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCYBPZlHLuGIhAA4dO9gHrVXmg0e3YvRS/5cg== 2024-03-16T20:44:38.119 INFO:teuthology.orchestra.run.smithi122.stdout:[mds.b] 2024-03-16T20:44:38.119 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCYBPZlxwYaLRAAY7PpU3RdRuEvXt9RpPEWiQ== 2024-03-16T20:44:38.119 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.4] 2024-03-16T20:44:38.119 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCpBPZlvYwKDhAAZ5t2Tj/h1Fwnqsto01FtSg== 2024-03-16T20:44:38.119 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.5] 2024-03-16T20:44:38.119 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCsBPZlr1gqEhAALvShoDQpoXGkzrJCAknq+A== 2024-03-16T20:44:38.119 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.6] 2024-03-16T20:44:38.120 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCvBPZlzlQuGBAAgqfkx5fzOYjnokkR+UPOpQ== 2024-03-16T20:44:38.120 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.7] 2024-03-16T20:44:38.120 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCyBPZlxHcNHhAAfl+lxu1il8hS0+tJdg6OXA== 2024-03-16T20:44:38.120 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-03-16T20:44:38.120 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCYBPZl3zQpMxAAhyRAHkPMirH6skmDMK55+A== 2024-03-16T20:44:38.120 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr.y] 2024-03-16T20:44:38.120 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCYBPZlvgfHHxAAeorpgzjZnG6znj3MHGlTvA== 2024-03-16T20:44:38.120 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.a] 2024-03-16T20:44:38.120 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCYBPZldiQwJxAAqp05o6yqMhy6pZoNHnDHKA== 2024-03-16T20:44:38.121 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.0] 2024-03-16T20:44:38.121 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCbBPZlS35pLxAAh2wHT8Aeinet1hO5A1ioyg== 2024-03-16T20:44:38.121 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.1] 2024-03-16T20:44:38.121 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCeBPZl+cEDNRAAPCKGGs2VdPWHsPvgdOnxIQ== 2024-03-16T20:44:38.121 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.2] 2024-03-16T20:44:38.121 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQChBPZlYcdGOxAAW+DGd7sDp76oQ7YgETtrBg== 2024-03-16T20:44:38.121 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.3] 2024-03-16T20:44:38.121 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQClBPZleeJnCRAAueEygVrApZN0Cc6L2q2kng== 2024-03-16T20:44:38.121 INFO:teuthology.orchestra.run.smithi195.stdout:[mgr.x] 2024-03-16T20:44:38.121 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCYBPZlHLuGIhAA4dO9gHrVXmg0e3YvRS/5cg== 2024-03-16T20:44:38.121 INFO:teuthology.orchestra.run.smithi195.stdout:[mds.b] 2024-03-16T20:44:38.121 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCYBPZlxwYaLRAAY7PpU3RdRuEvXt9RpPEWiQ== 2024-03-16T20:44:38.121 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.4] 2024-03-16T20:44:38.121 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCpBPZlvYwKDhAAZ5t2Tj/h1Fwnqsto01FtSg== 2024-03-16T20:44:38.121 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.5] 2024-03-16T20:44:38.121 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCsBPZlr1gqEhAALvShoDQpoXGkzrJCAknq+A== 2024-03-16T20:44:38.122 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.6] 2024-03-16T20:44:38.122 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCvBPZlzlQuGBAAgqfkx5fzOYjnokkR+UPOpQ== 2024-03-16T20:44:38.122 INFO:teuthology.orchestra.run.smithi195.stdout:[osd.7] 2024-03-16T20:44:38.122 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCyBPZlxHcNHhAAfl+lxu1il8hS0+tJdg6OXA== 2024-03-16T20:44:38.122 INFO:teuthology.orchestra.run.smithi195.stdout:[client.0] 2024-03-16T20:44:38.122 INFO:teuthology.orchestra.run.smithi195.stdout: key = AQCYBPZl3zQpMxAAhyRAHkPMirH6skmDMK55+A== 2024-03-16T20:44:38.123 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-16T20:44:38.165 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-16T20:44:38.211 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-16T20:44:38.253 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-16T20:44:38.298 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T20:44:38.340 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T20:44:38.382 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T20:44:38.425 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T20:44:38.465 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T20:44:38.509 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T20:44:38.548 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T20:44:38.592 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T20:44:38.636 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-16T20:44:38.681 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-03-16T20:44:38.727 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-16T20:44:38.729 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-03-16T20:44:38.814 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T20:44:38.817 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T20:44:38.896 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T20:44:38.899 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T20:44:38.977 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T20:44:38.985 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T20:44:39.062 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T20:44:39.065 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-03-16T20:44:39.150 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-16T20:44:39.152 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-03-16T20:44:39.242 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-03-16T20:44:39.243 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-03-16T20:44:39.255 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-16T20:44:39.351 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-03-16T20:44:39.361 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-03-16T20:44:39.374 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-16T20:44:39.468 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-03-16T20:44:39.478 DEBUG:teuthology.orchestra.run.smithi195:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-03-16T20:44:39.532 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-03-16T20:44:39.623 DEBUG:teuthology.orchestra.run.smithi195:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-03-16T20:44:39.634 DEBUG:teuthology.orchestra.run.smithi122:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-16T20:44:39.636 DEBUG:teuthology.orchestra.run.smithi195:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-03-16T20:44:39.681 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-03-16T20:44:39.681 INFO:tasks.ceph.mon.a:Restarting daemon 2024-03-16T20:44:39.681 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-03-16T20:44:39.685 INFO:tasks.ceph.mon.a:Started 2024-03-16T20:44:39.685 INFO:tasks.ceph.mon.b:Restarting daemon 2024-03-16T20:44:39.685 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-03-16T20:44:39.724 INFO:tasks.ceph.mon.b:Started 2024-03-16T20:44:39.724 INFO:tasks.ceph.mon.c:Restarting daemon 2024-03-16T20:44:39.724 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-03-16T20:44:39.726 INFO:tasks.ceph.mon.c:Started 2024-03-16T20:44:39.726 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-03-16T20:44:39.726 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-03-16T20:44:39.726 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-03-16T20:44:39.729 INFO:tasks.ceph.mgr.y:Started 2024-03-16T20:44:39.729 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-03-16T20:44:39.729 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-03-16T20:44:39.731 INFO:tasks.ceph.mgr.x:Started 2024-03-16T20:44:39.731 INFO:tasks.ceph:Setting crush tunables to default 2024-03-16T20:44:39.731 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd crush tunables default 2024-03-16T20:44:39.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4f2e5640 1 Processor -- start 2024-03-16T20:44:39.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4f2e5640 1 -- start start 2024-03-16T20:44:39.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4f2e5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee48141580 0x7fee480baac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:39.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4f2e5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fee480bb000 0x7fee4806cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:39.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4f2e5640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fee4806d180 0x7fee4814a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:39.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4f2e5640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee48147ff0 con 0x7fee4806d180 2024-03-16T20:44:39.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4f2e5640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee480bb460 con 0x7fee480bb000 2024-03-16T20:44:39.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4f2e5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee481479a0 con 0x7fee48141580 2024-03-16T20:44:39.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4d882640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fee4806d180 0x7fee4814a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:39.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4d882640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fee4806d180 0x7fee4814a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49290/0 (socket says 172.21.15.122:49290) 2024-03-16T20:44:39.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4d882640 1 -- 172.21.15.122:0/3389701474 learned_addr learned my addr 172.21.15.122:0/3389701474 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:44:39.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4d882640 1 -- 172.21.15.122:0/3389701474 >> v2:172.21.15.122:3300/0 conn(0x7fee4806d180 msgr2=0x7fee4814a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-16T20:44:39.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4d882640 1 -- 172.21.15.122:0/3389701474 >> v2:172.21.15.122:3300/0 conn(0x7fee4806d180 msgr2=0x7fee4814a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-16T20:44:39.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4d882640 1 --2- 172.21.15.122:0/3389701474 >> v2:172.21.15.122:3300/0 conn(0x7fee4806d180 0x7fee4814a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T20:44:39.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4d882640 1 --2- 172.21.15.122:0/3389701474 >> v2:172.21.15.122:3300/0 conn(0x7fee4806d180 0x7fee4814a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T20:44:39.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4d081640 1 --2- 172.21.15.122:0/3389701474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee48141580 0x7fee480baac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:39.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4c880640 1 --2- 172.21.15.122:0/3389701474 >> v2:172.21.15.195:3300/0 conn(0x7fee480bb000 0x7fee4806cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:39.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4d081640 1 -- 172.21.15.122:0/3389701474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee48141580 msgr2=0x7fee480baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-16T20:44:39.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4d081640 1 -- 172.21.15.122:0/3389701474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee48141580 msgr2=0x7fee480baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-16T20:44:39.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4d081640 1 --2- 172.21.15.122:0/3389701474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee48141580 0x7fee480baac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T20:44:39.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4d081640 1 --2- 172.21.15.122:0/3389701474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee48141580 0x7fee480baac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T20:44:39.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4c880640 1 -- 172.21.15.122:0/3389701474 >> v2:172.21.15.195:3300/0 conn(0x7fee480bb000 msgr2=0x7fee4806cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-16T20:44:39.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4c880640 1 -- 172.21.15.122:0/3389701474 >> v2:172.21.15.195:3300/0 conn(0x7fee480bb000 msgr2=0x7fee4806cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-16T20:44:39.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4c880640 1 --2- 172.21.15.122:0/3389701474 >> v2:172.21.15.195:3300/0 conn(0x7fee480bb000 0x7fee4806cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T20:44:39.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:39.847+0000 7fee4c880640 1 --2- 172.21.15.122:0/3389701474 >> v2:172.21.15.195:3300/0 conn(0x7fee480bb000 0x7fee4806cc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-16T20:44:40.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.051+0000 7fee4d882640 1 --2- 172.21.15.122:0/3389701474 >> v2:172.21.15.122:3300/0 conn(0x7fee4806d180 0x7fee4814a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:40.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.051+0000 7fee4d081640 1 --2- 172.21.15.122:0/3389701474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee48141580 0x7fee480baac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:40.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.051+0000 7fee4c880640 1 --2- 172.21.15.122:0/3389701474 >> v2:172.21.15.195:3300/0 conn(0x7fee480bb000 0x7fee4806cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:40.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.051+0000 7fee4d882640 1 -- 172.21.15.122:0/3389701474 >> v2:172.21.15.122:3300/0 conn(0x7fee4806d180 msgr2=0x7fee4814a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-03-16T20:44:40.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.051+0000 7fee4d882640 1 -- 172.21.15.122:0/3389701474 >> v2:172.21.15.122:3300/0 conn(0x7fee4806d180 msgr2=0x7fee4814a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-16T20:44:40.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.051+0000 7fee4d882640 1 --2- 172.21.15.122:0/3389701474 >> v2:172.21.15.122:3300/0 conn(0x7fee4806d180 0x7fee4814a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T20:44:40.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.051+0000 7fee4d882640 1 --2- 172.21.15.122:0/3389701474 >> v2:172.21.15.122:3300/0 conn(0x7fee4806d180 0x7fee4814a5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T20:44:40.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.051+0000 7fee4c880640 1 -- 172.21.15.122:0/3389701474 >> v2:172.21.15.195:3300/0 conn(0x7fee480bb000 msgr2=0x7fee4806cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-03-16T20:44:40.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.051+0000 7fee4d081640 1 -- 172.21.15.122:0/3389701474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee48141580 msgr2=0x7fee480baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-03-16T20:44:40.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.051+0000 7fee4c880640 1 -- 172.21.15.122:0/3389701474 >> v2:172.21.15.195:3300/0 conn(0x7fee480bb000 msgr2=0x7fee4806cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-16T20:44:40.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.051+0000 7fee4d081640 1 -- 172.21.15.122:0/3389701474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee48141580 msgr2=0x7fee480baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-03-16T20:44:40.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.051+0000 7fee4d081640 1 --2- 172.21.15.122:0/3389701474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee48141580 0x7fee480baac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T20:44:40.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.051+0000 7fee4c880640 1 --2- 172.21.15.122:0/3389701474 >> v2:172.21.15.195:3300/0 conn(0x7fee480bb000 0x7fee4806cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-16T20:44:40.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.051+0000 7fee4d081640 1 --2- 172.21.15.122:0/3389701474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee48141580 0x7fee480baac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T20:44:40.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.051+0000 7fee4c880640 1 --2- 172.21.15.122:0/3389701474 >> v2:172.21.15.195:3300/0 conn(0x7fee480bb000 0x7fee4806cc40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-16T20:44:40.261 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:40.258+0000 7ff351d72180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T20:44:40.294 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:40.291+0000 7fe73285d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-16T20:44:40.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.455+0000 7fee4d882640 1 --2- 172.21.15.122:0/3389701474 >> v2:172.21.15.122:3300/0 conn(0x7fee4806d180 0x7fee4814a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:40.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.455+0000 7fee4d081640 1 --2- 172.21.15.122:0/3389701474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee48141580 0x7fee480baac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:40.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.455+0000 7fee4c880640 1 --2- 172.21.15.122:0/3389701474 >> v2:172.21.15.195:3300/0 conn(0x7fee480bb000 0x7fee4806cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:40.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.455+0000 7fee4d882640 1 -- 172.21.15.122:0/3389701474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee48141580 msgr2=0x7fee480baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:40.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.455+0000 7fee4d882640 1 --2- 172.21.15.122:0/3389701474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee48141580 0x7fee480baac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:40.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.455+0000 7fee4d882640 1 -- 172.21.15.122:0/3389701474 >> v2:172.21.15.195:3300/0 conn(0x7fee480bb000 msgr2=0x7fee4806cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:40.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.455+0000 7fee4d882640 1 --2- 172.21.15.122:0/3389701474 >> v2:172.21.15.195:3300/0 conn(0x7fee480bb000 0x7fee4806cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:40.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.455+0000 7fee4d882640 1 -- 172.21.15.122:0/3389701474 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee48147c80 con 0x7fee4806d180 2024-03-16T20:44:40.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.459+0000 7fee4d882640 1 --2- 172.21.15.122:0/3389701474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee4806d180 0x7fee4814a5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee480a2c30 tx=0x7fee4001cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea6cfa0007720699 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:40.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.459+0000 7fee3ffff640 1 -- 172.21.15.122:0/3389701474 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee400a5ae0 con 0x7fee4806d180 2024-03-16T20:44:40.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.459+0000 7fee3ffff640 1 -- 172.21.15.122:0/3389701474 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee400a9070 con 0x7fee4806d180 2024-03-16T20:44:40.460 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.459+0000 7fee3ffff640 1 -- 172.21.15.122:0/3389701474 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee400af400 con 0x7fee4806d180 2024-03-16T20:44:40.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.459+0000 7fee4f2e5640 1 -- 172.21.15.122:0/3389701474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee4806d180 msgr2=0x7fee4814a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:40.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.459+0000 7fee4f2e5640 1 --2- 172.21.15.122:0/3389701474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee4806d180 0x7fee4814a5d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee480a2c30 tx=0x7fee4001cd50 comp rx=0 tx=0).stop 2024-03-16T20:44:40.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.459+0000 7fee4f2e5640 1 -- 172.21.15.122:0/3389701474 shutdown_connections 2024-03-16T20:44:40.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.459+0000 7fee4f2e5640 1 --2- 172.21.15.122:0/3389701474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee4806d180 0x7fee4814a5d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:40.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.459+0000 7fee4f2e5640 1 --2- 172.21.15.122:0/3389701474 >> v2:172.21.15.195:3300/0 conn(0x7fee480bb000 0x7fee4806cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:40.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.459+0000 7fee4f2e5640 1 --2- 172.21.15.122:0/3389701474 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee48141580 0x7fee480baac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:40.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.459+0000 7fee4f2e5640 1 -- 172.21.15.122:0/3389701474 >> 172.21.15.122:0/3389701474 conn(0x7fee480bca20 msgr2=0x7fee480bce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:40.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.459+0000 7fee4f2e5640 1 -- 172.21.15.122:0/3389701474 shutdown_connections 2024-03-16T20:44:40.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.459+0000 7fee4f2e5640 1 -- 172.21.15.122:0/3389701474 wait complete. 2024-03-16T20:44:40.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4f2e5640 1 Processor -- start 2024-03-16T20:44:40.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4f2e5640 1 -- start start 2024-03-16T20:44:40.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4f2e5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee4806d180 0x7fee48146820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:40.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4f2e5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee480bb000 0x7fee480b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:40.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4f2e5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee48141580 0x7fee4813a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:40.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4f2e5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee48147100 con 0x7fee48141580 2024-03-16T20:44:40.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4f2e5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee480a08a0 con 0x7fee4806d180 2024-03-16T20:44:40.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4f2e5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee480b3d20 con 0x7fee480bb000 2024-03-16T20:44:40.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4d882640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee48141580 0x7fee4813a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:40.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4d882640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee48141580 0x7fee4813a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49358/0 (socket says 172.21.15.122:49358) 2024-03-16T20:44:40.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4d882640 1 -- 172.21.15.122:0/252354650 learned_addr learned my addr 172.21.15.122:0/252354650 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:44:40.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4d081640 1 --2- 172.21.15.122:0/252354650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee4806d180 0x7fee48146820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:40.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4c880640 1 --2- 172.21.15.122:0/252354650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee480bb000 0x7fee480b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:40.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4d882640 1 -- 172.21.15.122:0/252354650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee480bb000 msgr2=0x7fee480b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:40.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4d882640 1 --2- 172.21.15.122:0/252354650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee480bb000 0x7fee480b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:40.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4d882640 1 -- 172.21.15.122:0/252354650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee4806d180 msgr2=0x7fee48146820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:40.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4d882640 1 --2- 172.21.15.122:0/252354650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee4806d180 0x7fee48146820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:40.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4d882640 1 -- 172.21.15.122:0/252354650 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee400097e0 con 0x7fee48141580 2024-03-16T20:44:40.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4d882640 1 --2- 172.21.15.122:0/252354650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee48141580 0x7fee4813a7a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fee4001d5f0 tx=0x7fee4001b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:40.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee3dffb640 1 -- 172.21.15.122:0/252354650 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee400ba880 con 0x7fee48141580 2024-03-16T20:44:40.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee3dffb640 1 -- 172.21.15.122:0/252354650 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee400ba9e0 con 0x7fee48141580 2024-03-16T20:44:40.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4f2e5640 1 -- 172.21.15.122:0/252354650 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee480b4bc0 con 0x7fee48141580 2024-03-16T20:44:40.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee3dffb640 1 -- 172.21.15.122:0/252354650 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee400bad50 con 0x7fee48141580 2024-03-16T20:44:40.468 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee4f2e5640 1 -- 172.21.15.122:0/252354650 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee480a0d80 con 0x7fee48141580 2024-03-16T20:44:40.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee3dffb640 1 -- 172.21.15.122:0/252354650 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fee400c5460 con 0x7fee48141580 2024-03-16T20:44:40.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.463+0000 7fee3dffb640 1 -- 172.21.15.122:0/252354650 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fee400af520 con 0x7fee48141580 2024-03-16T20:44:40.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.467+0000 7fee4f2e5640 1 -- 172.21.15.122:0/252354650 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee480a0a60 con 0x7fee48141580 2024-03-16T20:44:40.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.467+0000 7fee3dffb640 1 -- 172.21.15.122:0/252354650 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fee400ab070 con 0x7fee48141580 2024-03-16T20:44:40.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.531+0000 7fee4f2e5640 1 -- 172.21.15.122:0/252354650 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fee48074bb0 con 0x7fee48141580 2024-03-16T20:44:40.536 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.535+0000 7fee3dffb640 1 -- 172.21.15.122:0/252354650 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fee400c58a0 con 0x7fee48141580 2024-03-16T20:44:40.536 INFO:teuthology.orchestra.run.smithi122.stderr:adjusted tunables profile to default 2024-03-16T20:44:40.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.535+0000 7fee4f2e5640 1 -- 172.21.15.122:0/252354650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee48141580 msgr2=0x7fee4813a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:40.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.535+0000 7fee4f2e5640 1 --2- 172.21.15.122:0/252354650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee48141580 0x7fee4813a7a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fee4001d5f0 tx=0x7fee4001b590 comp rx=0 tx=0).stop 2024-03-16T20:44:40.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.535+0000 7fee4f2e5640 1 -- 172.21.15.122:0/252354650 shutdown_connections 2024-03-16T20:44:40.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.535+0000 7fee4f2e5640 1 --2- 172.21.15.122:0/252354650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fee48141580 0x7fee4813a7a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:40.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.535+0000 7fee4f2e5640 1 --2- 172.21.15.122:0/252354650 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fee480bb000 0x7fee480b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:40.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.535+0000 7fee4f2e5640 1 --2- 172.21.15.122:0/252354650 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fee4806d180 0x7fee48146820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:40.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.535+0000 7fee4f2e5640 1 -- 172.21.15.122:0/252354650 >> 172.21.15.122:0/252354650 conn(0x7fee480bca20 msgr2=0x7fee480b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:40.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.535+0000 7fee4f2e5640 1 -- 172.21.15.122:0/252354650 shutdown_connections 2024-03-16T20:44:40.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:40.535+0000 7fee4f2e5640 1 -- 172.21.15.122:0/252354650 wait complete. 2024-03-16T20:44:40.554 INFO:tasks.ceph:check_enable_crimson: False 2024-03-16T20:44:40.554 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-03-16T20:44:40.554 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:40.554 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-03-16T20:44:40.564 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:40.564 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-03-16T20:44:40.616 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:40.617 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-03-16T20:44:40.670 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:44:40.671 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-03-16T20:44:40.721 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:40.721 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-03-16T20:44:40.727 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:40.726+0000 7ff351d72180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T20:44:40.734 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:40.735 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-03-16T20:44:40.768 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:40.767+0000 7fe73285d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-16T20:44:40.789 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:40.790 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-03-16T20:44:40.841 DEBUG:teuthology.orchestra.run.smithi195:> set -ex 2024-03-16T20:44:40.841 DEBUG:teuthology.orchestra.run.smithi195:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-03-16T20:44:40.895 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new bf3aff4c-c53f-4a6e-a538-7550585bb0a0 0 2024-03-16T20:44:40.911 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:40.910+0000 7ff351d72180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T20:44:40.971 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:40.967+0000 7fe73285d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-16T20:44:41.029 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.026+0000 7f07e894d640 1 Processor -- start 2024-03-16T20:44:41.029 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.026+0000 7f07e894d640 1 -- start start 2024-03-16T20:44:41.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.026+0000 7f07e894d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f07e4144290 0x7f07e406ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.026+0000 7f07e894d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e40b90c0 0x7f07e40b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.026+0000 7f07e894d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e406daf0 con 0x7f07e4144c00 2024-03-16T20:44:41.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.026+0000 7f07e894d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e40b1390 con 0x7f07e4144290 2024-03-16T20:44:41.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.026+0000 7f07e894d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e406dd40 con 0x7f07e40b90c0 2024-03-16T20:44:41.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e1d9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e40b90c0 0x7f07e40b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e259c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f07e4144290 0x7f07e406ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.031 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e2d9d640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f07e4144c00 0x7f07e413df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.195:59498/0 (socket says 172.21.15.195:59498) 2024-03-16T20:44:41.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e2d9d640 1 -- 172.21.15.195:0/1520660802 learned_addr learned my addr 172.21.15.195:0/1520660802 (peer_addr_for_me v1:172.21.15.195:0/0) 2024-03-16T20:44:41.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e259c640 1 -- 172.21.15.195:0/1520660802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e40b90c0 msgr2=0x7f07e40b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:41.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e259c640 1 --2- 172.21.15.195:0/1520660802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e40b90c0 0x7f07e40b7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e259c640 1 -- 172.21.15.195:0/1520660802 >> v1:172.21.15.195:6789/0 conn(0x7f07e4144c00 legacy=0x7f07e413df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.032 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e259c640 1 -- 172.21.15.195:0/1520660802 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e413e690 con 0x7f07e4144290 2024-03-16T20:44:41.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e259c640 1 --2- 172.21.15.195:0/1520660802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07e4144290 0x7f07e406ed70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f07d40097c0 tx=0x7f07d401e590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=324c764c2aa3ccdc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:41.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e159a640 1 -- 172.21.15.195:0/1520660802 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d40ac620 con 0x7f07e4144290 2024-03-16T20:44:41.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e159a640 1 -- 172.21.15.195:0/1520660802 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d40ac780 con 0x7f07e4144290 2024-03-16T20:44:41.033 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e159a640 1 -- 172.21.15.195:0/1520660802 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d40acb10 con 0x7f07e4144290 2024-03-16T20:44:41.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e894d640 1 -- 172.21.15.195:0/1520660802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07e4144290 msgr2=0x7f07e406ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e894d640 1 --2- 172.21.15.195:0/1520660802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07e4144290 0x7f07e406ed70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f07d40097c0 tx=0x7f07d401e590 comp rx=0 tx=0).stop 2024-03-16T20:44:41.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e894d640 1 -- 172.21.15.195:0/1520660802 shutdown_connections 2024-03-16T20:44:41.034 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e894d640 1 --2- 172.21.15.195:0/1520660802 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e40b90c0 0x7f07e40b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.035 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e894d640 1 --2- 172.21.15.195:0/1520660802 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07e4144290 0x7f07e406ed70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.035 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e894d640 1 -- 172.21.15.195:0/1520660802 >> 172.21.15.195:0/1520660802 conn(0x7f07e4070d30 msgr2=0x7f07e4071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:41.035 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e894d640 1 -- 172.21.15.195:0/1520660802 shutdown_connections 2024-03-16T20:44:41.035 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.030+0000 7f07e894d640 1 -- 172.21.15.195:0/1520660802 wait complete. 2024-03-16T20:44:41.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e894d640 1 Processor -- start 2024-03-16T20:44:41.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e894d640 1 -- start start 2024-03-16T20:44:41.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e894d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07e40b90c0 0x7f07e413ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.037 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e894d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e4144290 0x7f07e40b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e894d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07e4144c00 0x7f07e41e2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e894d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40b5a70 con 0x7f07e40b90c0 2024-03-16T20:44:41.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e894d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40b3480 con 0x7f07e4144c00 2024-03-16T20:44:41.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e894d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e40b5c60 con 0x7f07e4144290 2024-03-16T20:44:41.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e1d9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e4144290 0x7f07e40b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e2d9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07e4144c00 0x7f07e41e2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e2d9d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07e4144c00 0x7f07e41e2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:39420/0 (socket says 172.21.15.195:39420) 2024-03-16T20:44:41.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e2d9d640 1 -- 172.21.15.195:0/1955257656 learned_addr learned my addr 172.21.15.195:0/1955257656 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-16T20:44:41.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e259c640 1 --2- 172.21.15.195:0/1955257656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07e40b90c0 0x7f07e413ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e1d9b640 1 -- 172.21.15.195:0/1955257656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07e4144c00 msgr2=0x7f07e41e2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e1d9b640 1 --2- 172.21.15.195:0/1955257656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07e4144c00 0x7f07e41e2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e1d9b640 1 -- 172.21.15.195:0/1955257656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07e40b90c0 msgr2=0x7f07e413ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.038 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e1d9b640 1 --2- 172.21.15.195:0/1955257656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07e40b90c0 0x7f07e413ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.039 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e1d9b640 1 -- 172.21.15.195:0/1955257656 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d4009420 con 0x7f07e4144290 2024-03-16T20:44:41.039 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e2d9d640 1 --2- 172.21.15.195:0/1955257656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07e4144c00 0x7f07e41e2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:44:41.039 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e1d9b640 1 --2- 172.21.15.195:0/1955257656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e4144290 0x7f07e40b5530 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f07d800e870 tx=0x7f07d800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:41.039 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07d2ffd640 1 -- 172.21.15.195:0/1955257656 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d800ee60 con 0x7f07e4144290 2024-03-16T20:44:41.039 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07d2ffd640 1 -- 172.21.15.195:0/1955257656 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d800c940 con 0x7f07e4144290 2024-03-16T20:44:41.039 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07d2ffd640 1 -- 172.21.15.195:0/1955257656 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d800ccd0 con 0x7f07e4144290 2024-03-16T20:44:41.039 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e894d640 1 -- 172.21.15.195:0/1955257656 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e40b5f40 con 0x7f07e4144290 2024-03-16T20:44:41.039 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e894d640 1 -- 172.21.15.195:0/1955257656 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e41452c0 con 0x7f07e4144290 2024-03-16T20:44:41.039 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07d2ffd640 1 -- 172.21.15.195:0/1955257656 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f07d80059a0 con 0x7f07e4144290 2024-03-16T20:44:41.039 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07d2ffd640 1 -- 172.21.15.195:0/1955257656 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f07d80137b0 con 0x7f07e4144290 2024-03-16T20:44:41.039 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.034+0000 7f07e894d640 1 -- 172.21.15.195:0/1955257656 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e413d750 con 0x7f07e4144290 2024-03-16T20:44:41.047 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.046+0000 7f07d2ffd640 1 -- 172.21.15.195:0/1955257656 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f07e413d750 con 0x7f07e4144290 2024-03-16T20:44:41.116 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.114+0000 7f07e894d640 1 -- 172.21.15.195:0/1955257656 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd new", "uuid": "bf3aff4c-c53f-4a6e-a538-7550585bb0a0", "id": 0} v 0) v1 -- 0x7f07e40ba6b0 con 0x7f07e4144290 2024-03-16T20:44:41.124 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.122+0000 7f07d2ffd640 1 -- 172.21.15.195:0/1955257656 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "bf3aff4c-c53f-4a6e-a538-7550585bb0a0", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f07d8021060 con 0x7f07e4144290 2024-03-16T20:44:41.124 INFO:teuthology.orchestra.run.smithi195.stdout:0 2024-03-16T20:44:41.125 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.122+0000 7f07e894d640 1 -- 172.21.15.195:0/1955257656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e4144290 msgr2=0x7f07e40b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.125 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.122+0000 7f07e894d640 1 --2- 172.21.15.195:0/1955257656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e4144290 0x7f07e40b5530 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f07d800e870 tx=0x7f07d800ed30 comp rx=0 tx=0).stop 2024-03-16T20:44:41.126 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.122+0000 7f07e894d640 1 -- 172.21.15.195:0/1955257656 shutdown_connections 2024-03-16T20:44:41.126 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.122+0000 7f07e894d640 1 --2- 172.21.15.195:0/1955257656 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f07e4144c00 0x7f07e41e2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.126 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.122+0000 7f07e894d640 1 --2- 172.21.15.195:0/1955257656 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07e4144290 0x7f07e40b5530 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.126 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.122+0000 7f07e894d640 1 --2- 172.21.15.195:0/1955257656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f07e40b90c0 0x7f07e413ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.126 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.122+0000 7f07e894d640 1 -- 172.21.15.195:0/1955257656 >> 172.21.15.195:0/1955257656 conn(0x7f07e4070d30 msgr2=0x7f07e40b8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:41.126 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.122+0000 7f07e894d640 1 -- 172.21.15.195:0/1955257656 shutdown_connections 2024-03-16T20:44:41.126 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.122+0000 7f07e894d640 1 -- 172.21.15.195:0/1955257656 wait complete. 2024-03-16T20:44:41.140 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b 1 2024-03-16T20:44:41.251 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.250+0000 7fac55101640 1 Processor -- start 2024-03-16T20:44:41.252 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.250+0000 7fac55101640 1 -- start start 2024-03-16T20:44:41.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.250+0000 7fac55101640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fac5006e030 0x7fac50144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.250+0000 7fac55101640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fac50145510 0x7fac5013ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.250+0000 7fac55101640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac5013f740 con 0x7fac5013f2d0 2024-03-16T20:44:41.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.250+0000 7fac55101640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac5009a130 con 0x7fac50145510 2024-03-16T20:44:41.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.250+0000 7fac55101640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac5006e410 con 0x7fac5006e030 2024-03-16T20:44:41.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.250+0000 7fac4ed9d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fac5006e030 0x7fac50144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.250+0000 7fac4ed9d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fac5006e030 0x7fac50144fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:39426/0 (socket says 172.21.15.195:39426) 2024-03-16T20:44:41.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.250+0000 7fac4ed9d640 1 -- 172.21.15.195:0/3271532725 learned_addr learned my addr 172.21.15.195:0/3271532725 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-16T20:44:41.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.250+0000 7fac4f59e640 1 --1- 172.21.15.195:0/3271532725 >> v1:172.21.15.195:6789/0 conn(0x7fac5013f2d0 0x7fac50148530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.195:59508/0 (socket says 172.21.15.195:59508) 2024-03-16T20:44:41.253 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.250+0000 7fac4e59c640 1 --2- 172.21.15.195:0/3271532725 >> v2:172.21.15.122:3300/0 conn(0x7fac50145510 0x7fac5013ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.254 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.250+0000 7fac4ed9d640 1 -- 172.21.15.195:0/3271532725 >> v2:172.21.15.122:3300/0 conn(0x7fac50145510 msgr2=0x7fac5013ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:41.254 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.250+0000 7fac4ed9d640 1 --2- 172.21.15.195:0/3271532725 >> v2:172.21.15.122:3300/0 conn(0x7fac50145510 0x7fac5013ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.254 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.250+0000 7fac4ed9d640 1 -- 172.21.15.195:0/3271532725 >> v1:172.21.15.195:6789/0 conn(0x7fac5013f2d0 legacy=0x7fac50148530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.254 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.250+0000 7fac4ed9d640 1 -- 172.21.15.195:0/3271532725 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac50099de0 con 0x7fac5006e030 2024-03-16T20:44:41.256 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.254+0000 7fac4ed9d640 1 --2- 172.21.15.195:0/3271532725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac5006e030 0x7fac50144fd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fac40009870 tx=0x7fac4001c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a303383b44b72674 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:41.256 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.254+0000 7fac4dd9b640 1 -- 172.21.15.195:0/3271532725 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac400ab600 con 0x7fac5006e030 2024-03-16T20:44:41.256 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.254+0000 7fac4dd9b640 1 -- 172.21.15.195:0/3271532725 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac400a7050 con 0x7fac5006e030 2024-03-16T20:44:41.256 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.254+0000 7fac4dd9b640 1 -- 172.21.15.195:0/3271532725 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac400ab980 con 0x7fac5006e030 2024-03-16T20:44:41.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.254+0000 7fac55101640 1 -- 172.21.15.195:0/3271532725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac5006e030 msgr2=0x7fac50144fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.254+0000 7fac55101640 1 --2- 172.21.15.195:0/3271532725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac5006e030 0x7fac50144fd0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fac40009870 tx=0x7fac4001c490 comp rx=0 tx=0).stop 2024-03-16T20:44:41.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.254+0000 7fac55101640 1 -- 172.21.15.195:0/3271532725 shutdown_connections 2024-03-16T20:44:41.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.254+0000 7fac55101640 1 --2- 172.21.15.195:0/3271532725 >> v2:172.21.15.122:3300/0 conn(0x7fac50145510 0x7fac5013ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.254+0000 7fac55101640 1 --2- 172.21.15.195:0/3271532725 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac5006e030 0x7fac50144fd0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.254+0000 7fac55101640 1 -- 172.21.15.195:0/3271532725 >> 172.21.15.195:0/3271532725 conn(0x7fac5006d2b0 msgr2=0x7fac5006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:41.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.254+0000 7fac55101640 1 -- 172.21.15.195:0/3271532725 shutdown_connections 2024-03-16T20:44:41.257 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.254+0000 7fac55101640 1 -- 172.21.15.195:0/3271532725 wait complete. 2024-03-16T20:44:41.258 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.254+0000 7fac55101640 1 Processor -- start 2024-03-16T20:44:41.258 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.254+0000 7fac55101640 1 -- start start 2024-03-16T20:44:41.260 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac55101640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac5006e030 0x7fac5013ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.260 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac55101640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac5013f2d0 0x7fac50070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.260 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac55101640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac50145510 0x7fac50071400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.260 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac55101640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac50150300 con 0x7fac50145510 2024-03-16T20:44:41.261 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac55101640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac500ae410 con 0x7fac5013f2d0 2024-03-16T20:44:41.261 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac55101640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac500706e0 con 0x7fac5006e030 2024-03-16T20:44:41.261 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac4e59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac5013f2d0 0x7fac50070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.261 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac4ed9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac5006e030 0x7fac5013ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.261 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac4e59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac5013f2d0 0x7fac50070ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:39440/0 (socket says 172.21.15.195:39440) 2024-03-16T20:44:41.261 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac4e59c640 1 -- 172.21.15.195:0/979215566 learned_addr learned my addr 172.21.15.195:0/979215566 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-16T20:44:41.261 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac4f59e640 1 --2- 172.21.15.195:0/979215566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac50145510 0x7fac50071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.261 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac4e59c640 1 -- 172.21.15.195:0/979215566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac5006e030 msgr2=0x7fac5013ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.261 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac4e59c640 1 --2- 172.21.15.195:0/979215566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac5006e030 0x7fac5013ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.261 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac4e59c640 1 -- 172.21.15.195:0/979215566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac50145510 msgr2=0x7fac50071400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.261 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac4e59c640 1 --2- 172.21.15.195:0/979215566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac50145510 0x7fac50071400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.261 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac4e59c640 1 -- 172.21.15.195:0/979215566 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac400094d0 con 0x7fac5013f2d0 2024-03-16T20:44:41.261 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac4ed9d640 1 --2- 172.21.15.195:0/979215566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac5006e030 0x7fac5013ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:44:41.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac4f59e640 1 --2- 172.21.15.195:0/979215566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac50145510 0x7fac50071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:44:41.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac4e59c640 1 --2- 172.21.15.195:0/979215566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac5013f2d0 0x7fac50070ec0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fac4400c7f0 tx=0x7fac4400ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:41.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac377fe640 1 -- 172.21.15.195:0/979215566 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac44004230 con 0x7fac5013f2d0 2024-03-16T20:44:41.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac377fe640 1 -- 172.21.15.195:0/979215566 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac44004390 con 0x7fac5013f2d0 2024-03-16T20:44:41.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac55101640 1 -- 172.21.15.195:0/979215566 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5014ff90 con 0x7fac5013f2d0 2024-03-16T20:44:41.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac377fe640 1 -- 172.21.15.195:0/979215566 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac44003860 con 0x7fac5013f2d0 2024-03-16T20:44:41.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac55101640 1 -- 172.21.15.195:0/979215566 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac500adf80 con 0x7fac5013f2d0 2024-03-16T20:44:41.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac377fe640 1 -- 172.21.15.195:0/979215566 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fac44007620 con 0x7fac5013f2d0 2024-03-16T20:44:41.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac377fe640 1 -- 172.21.15.195:0/979215566 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fac44019cc0 con 0x7fac5013f2d0 2024-03-16T20:44:41.262 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.258+0000 7fac55101640 1 -- 172.21.15.195:0/979215566 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac500ae260 con 0x7fac5013f2d0 2024-03-16T20:44:41.263 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.262+0000 7fac377fe640 1 -- 172.21.15.195:0/979215566 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fac44038dd0 con 0x7fac5013f2d0 2024-03-16T20:44:41.330 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.326+0000 7fac55101640 1 -- 172.21.15.195:0/979215566 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b", "id": 1} v 0) v1 -- 0x7fac50074bb0 con 0x7fac5013f2d0 2024-03-16T20:44:41.336 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.334+0000 7fac377fe640 1 -- 172.21.15.195:0/979215566 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fac44027030 con 0x7fac5013f2d0 2024-03-16T20:44:41.336 INFO:teuthology.orchestra.run.smithi195.stdout:1 2024-03-16T20:44:41.337 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.334+0000 7fac55101640 1 -- 172.21.15.195:0/979215566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac5013f2d0 msgr2=0x7fac50070ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.337 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.334+0000 7fac55101640 1 --2- 172.21.15.195:0/979215566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac5013f2d0 0x7fac50070ec0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fac4400c7f0 tx=0x7fac4400ccb0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.337 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.334+0000 7fac55101640 1 -- 172.21.15.195:0/979215566 shutdown_connections 2024-03-16T20:44:41.338 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.334+0000 7fac55101640 1 --2- 172.21.15.195:0/979215566 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac50145510 0x7fac50071400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.338 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.334+0000 7fac55101640 1 --2- 172.21.15.195:0/979215566 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac5013f2d0 0x7fac50070ec0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.338 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.334+0000 7fac55101640 1 --2- 172.21.15.195:0/979215566 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac5006e030 0x7fac5013ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.338 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.334+0000 7fac55101640 1 -- 172.21.15.195:0/979215566 >> 172.21.15.195:0/979215566 conn(0x7fac5006d2b0 msgr2=0x7fac500b81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:41.338 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.334+0000 7fac55101640 1 -- 172.21.15.195:0/979215566 shutdown_connections 2024-03-16T20:44:41.338 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.334+0000 7fac55101640 1 -- 172.21.15.195:0/979215566 wait complete. 2024-03-16T20:44:41.353 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 367a357b-c342-44c6-8155-fdab43bfb947 2 2024-03-16T20:44:41.461 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.458+0000 7f8033e5e640 1 Processor -- start 2024-03-16T20:44:41.461 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.458+0000 7f8033e5e640 1 -- start start 2024-03-16T20:44:41.462 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.458+0000 7f8033e5e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f802c0b7540 0x7f802c0b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.463 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.458+0000 7f8033e5e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f802c0b7d40 0x7f802c13ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.463 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.458+0000 7f8033e5e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f802c0b6740 con 0x7f802c13f460 2024-03-16T20:44:41.463 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.458+0000 7f8033e5e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802c0b23d0 con 0x7f802c0b7d40 2024-03-16T20:44:41.463 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.458+0000 7f8033e5e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0b7110 con 0x7f802c0b7540 2024-03-16T20:44:41.463 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.458+0000 7f80313f9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f802c0b7d40 0x7f802c13ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.463 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.458+0000 7f80313f9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f802c0b7d40 0x7f802c13ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:39442/0 (socket says 172.21.15.195:39442) 2024-03-16T20:44:41.463 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.458+0000 7f80313f9640 1 -- 172.21.15.195:0/76314206 learned_addr learned my addr 172.21.15.195:0/76314206 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-16T20:44:41.463 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.458+0000 7f8031bfa640 1 --2- 172.21.15.195:0/76314206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f802c0b7540 0x7f802c0b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.463 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.458+0000 7f80313f9640 1 -- 172.21.15.195:0/76314206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f802c0b7540 msgr2=0x7f802c0b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:41.463 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.458+0000 7f80313f9640 1 --2- 172.21.15.195:0/76314206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f802c0b7540 0x7f802c0b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.463 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.458+0000 7f80313f9640 1 -- 172.21.15.195:0/76314206 >> v1:172.21.15.122:6789/0 conn(0x7f802c13f460 legacy=0x7f802c0b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:41.463 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.458+0000 7f80313f9640 1 -- 172.21.15.195:0/76314206 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c0b7920 con 0x7f802c0b7d40 2024-03-16T20:44:41.463 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.458+0000 7f8031bfa640 1 --2- 172.21.15.195:0/76314206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f802c0b7540 0x7f802c0b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:44:41.464 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.462+0000 7f80313f9640 1 --2- 172.21.15.195:0/76314206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f802c0b7d40 0x7f802c13ef20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f80200098a0 tx=0x7f802001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecb7c89d78cc5afa server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:41.464 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.462+0000 7f8030bf8640 1 -- 172.21.15.195:0/76314206 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80200aa5f0 con 0x7f802c0b7d40 2024-03-16T20:44:41.465 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.462+0000 7f8030bf8640 1 -- 172.21.15.195:0/76314206 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80200aa750 con 0x7f802c0b7d40 2024-03-16T20:44:41.465 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.462+0000 7f8030bf8640 1 -- 172.21.15.195:0/76314206 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80200aaae0 con 0x7f802c0b7d40 2024-03-16T20:44:41.466 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.462+0000 7f8033e5e640 1 -- 172.21.15.195:0/76314206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f802c0b7d40 msgr2=0x7f802c13ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.466 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.462+0000 7f8033e5e640 1 --2- 172.21.15.195:0/76314206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f802c0b7d40 0x7f802c13ef20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f80200098a0 tx=0x7f802001c4d0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.466 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.462+0000 7f8033e5e640 1 -- 172.21.15.195:0/76314206 shutdown_connections 2024-03-16T20:44:41.466 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.462+0000 7f8033e5e640 1 --2- 172.21.15.195:0/76314206 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f802c0b7d40 0x7f802c13ef20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.466 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.462+0000 7f8033e5e640 1 --2- 172.21.15.195:0/76314206 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f802c0b7540 0x7f802c0b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.466 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.462+0000 7f8033e5e640 1 -- 172.21.15.195:0/76314206 >> 172.21.15.195:0/76314206 conn(0x7f802c06fc00 msgr2=0x7f802c070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:41.466 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.462+0000 7f8033e5e640 1 -- 172.21.15.195:0/76314206 shutdown_connections 2024-03-16T20:44:41.466 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.462+0000 7f8033e5e640 1 -- 172.21.15.195:0/76314206 wait complete. 2024-03-16T20:44:41.468 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.462+0000 7f8033e5e640 1 Processor -- start 2024-03-16T20:44:41.469 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.462+0000 7f8033e5e640 1 -- start start 2024-03-16T20:44:41.469 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8033e5e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f802c0b7540 0x7f802c13cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.469 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8033e5e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f802c0b7d40 0x7f802c13a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.469 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8033e5e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f802c13f460 0x7f802c06d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.469 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8033e5e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c13ce30 con 0x7f802c13f460 2024-03-16T20:44:41.469 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8033e5e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c13cbe0 con 0x7f802c0b7540 2024-03-16T20:44:41.469 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8033e5e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c1dfa30 con 0x7f802c0b7d40 2024-03-16T20:44:41.469 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8031bfa640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f802c0b7540 0x7f802c13cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.469 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f80313f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f802c0b7d40 0x7f802c13a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.469 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8031bfa640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f802c0b7540 0x7f802c13cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:39458/0 (socket says 172.21.15.195:39458) 2024-03-16T20:44:41.469 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8031bfa640 1 -- 172.21.15.195:0/3605745752 learned_addr learned my addr 172.21.15.195:0/3605745752 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-16T20:44:41.469 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f80323fb640 1 --2- 172.21.15.195:0/3605745752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f802c13f460 0x7f802c06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.469 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8031bfa640 1 -- 172.21.15.195:0/3605745752 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f802c0b7d40 msgr2=0x7f802c13a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.469 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8031bfa640 1 --2- 172.21.15.195:0/3605745752 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f802c0b7d40 0x7f802c13a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8031bfa640 1 -- 172.21.15.195:0/3605745752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f802c13f460 msgr2=0x7f802c06d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8031bfa640 1 --2- 172.21.15.195:0/3605745752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f802c13f460 0x7f802c06d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8031bfa640 1 -- 172.21.15.195:0/3605745752 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8028009420 con 0x7f802c0b7540 2024-03-16T20:44:41.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f80323fb640 1 --2- 172.21.15.195:0/3605745752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f802c13f460 0x7f802c06d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:44:41.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f80313f9640 1 --2- 172.21.15.195:0/3605745752 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f802c0b7d40 0x7f802c13a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:44:41.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8031bfa640 1 --2- 172.21.15.195:0/3605745752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f802c0b7540 0x7f802c13cfa0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f802800ebe0 tx=0x7f802800c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:41.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f801e7fc640 1 -- 172.21.15.195:0/3605745752 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802800eee0 con 0x7f802c0b7540 2024-03-16T20:44:41.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f801e7fc640 1 -- 172.21.15.195:0/3605745752 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802800cc30 con 0x7f802c0b7540 2024-03-16T20:44:41.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8033e5e640 1 -- 172.21.15.195:0/3605745752 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80200094d0 con 0x7f802c0b7540 2024-03-16T20:44:41.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f801e7fc640 1 -- 172.21.15.195:0/3605745752 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028013470 con 0x7f802c0b7540 2024-03-16T20:44:41.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8033e5e640 1 -- 172.21.15.195:0/3605745752 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c144580 con 0x7f802c0b7540 2024-03-16T20:44:41.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f801e7fc640 1 -- 172.21.15.195:0/3605745752 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f80280059a0 con 0x7f802c0b7540 2024-03-16T20:44:41.470 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f801e7fc640 1 -- 172.21.15.195:0/3605745752 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f8028006440 con 0x7f802c0b7540 2024-03-16T20:44:41.471 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.466+0000 7f8033e5e640 1 -- 172.21.15.195:0/3605745752 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c13e520 con 0x7f802c0b7540 2024-03-16T20:44:41.471 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.470+0000 7f801e7fc640 1 -- 172.21.15.195:0/3605745752 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f8028021060 con 0x7f802c0b7540 2024-03-16T20:44:41.486 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:41.482+0000 7ff351d72180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T20:44:41.523 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:41.519+0000 7fe73285d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-16T20:44:41.538 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.534+0000 7f8033e5e640 1 -- 172.21.15.195:0/3605745752 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "367a357b-c342-44c6-8155-fdab43bfb947", "id": 2} v 0) v1 -- 0x7f802c144770 con 0x7f802c0b7540 2024-03-16T20:44:41.543 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:41.542+0000 7ff351d72180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T20:44:41.544 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.542+0000 7f801e7fc640 1 -- 172.21.15.195:0/3605745752 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "367a357b-c342-44c6-8155-fdab43bfb947", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8028017070 con 0x7f802c0b7540 2024-03-16T20:44:41.544 INFO:teuthology.orchestra.run.smithi195.stdout:2 2024-03-16T20:44:41.546 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.542+0000 7f8033e5e640 1 -- 172.21.15.195:0/3605745752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f802c0b7540 msgr2=0x7f802c13cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.546 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.542+0000 7f8033e5e640 1 --2- 172.21.15.195:0/3605745752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f802c0b7540 0x7f802c13cfa0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f802800ebe0 tx=0x7f802800c520 comp rx=0 tx=0).stop 2024-03-16T20:44:41.546 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.542+0000 7f8033e5e640 1 -- 172.21.15.195:0/3605745752 shutdown_connections 2024-03-16T20:44:41.546 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.542+0000 7f8033e5e640 1 --2- 172.21.15.195:0/3605745752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f802c13f460 0x7f802c06d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.546 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.542+0000 7f8033e5e640 1 --2- 172.21.15.195:0/3605745752 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f802c0b7d40 0x7f802c13a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.547 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.542+0000 7f8033e5e640 1 --2- 172.21.15.195:0/3605745752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f802c0b7540 0x7f802c13cfa0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.547 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.542+0000 7f8033e5e640 1 -- 172.21.15.195:0/3605745752 >> 172.21.15.195:0/3605745752 conn(0x7f802c06fc00 msgr2=0x7f802c074320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:41.547 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.542+0000 7f8033e5e640 1 -- 172.21.15.195:0/3605745752 shutdown_connections 2024-03-16T20:44:41.547 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.542+0000 7f8033e5e640 1 -- 172.21.15.195:0/3605745752 wait complete. 2024-03-16T20:44:41.561 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 3587f37a-c824-4dea-a154-297942ff1e1f 3 2024-03-16T20:44:41.577 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:41.575+0000 7fe73285d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-16T20:44:41.599 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:41.598+0000 7ff351d72180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T20:44:41.629 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:41.627+0000 7fe73285d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-16T20:44:41.674 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.670+0000 7fe8c4886640 1 Processor -- start 2024-03-16T20:44:41.674 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.670+0000 7fe8c4886640 1 -- start start 2024-03-16T20:44:41.675 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.674+0000 7fe8c4886640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe8c006c300 0x7fe8c00b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.675 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.674+0000 7fe8c4886640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0141730 0x7fe8c0147e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.675 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.674+0000 7fe8c4886640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c0147510 con 0x7fe8c01479a0 2024-03-16T20:44:41.676 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.674+0000 7fe8c4886640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c00b5250 con 0x7fe8c006c300 2024-03-16T20:44:41.676 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.674+0000 7fe8c4886640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c0147290 con 0x7fe8c0141730 2024-03-16T20:44:41.676 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.674+0000 7fe8be59c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe8c006c300 0x7fe8c00b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.676 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.674+0000 7fe8bed9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0141730 0x7fe8c0147e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.676 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.674+0000 7fe8be59c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe8c006c300 0x7fe8c00b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:39462/0 (socket says 172.21.15.195:39462) 2024-03-16T20:44:41.676 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.674+0000 7fe8be59c640 1 -- 172.21.15.195:0/472713801 learned_addr learned my addr 172.21.15.195:0/472713801 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-16T20:44:41.676 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.674+0000 7fe8be59c640 1 -- 172.21.15.195:0/472713801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0141730 msgr2=0x7fe8c0147e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:41.676 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.674+0000 7fe8be59c640 1 --2- 172.21.15.195:0/472713801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0141730 0x7fe8c0147e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.676 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.674+0000 7fe8be59c640 1 -- 172.21.15.195:0/472713801 >> v1:172.21.15.122:6789/0 conn(0x7fe8c01479a0 legacy=0x7fe8c0141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:41.676 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.674+0000 7fe8be59c640 1 -- 172.21.15.195:0/472713801 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c006d5e0 con 0x7fe8c006c300 2024-03-16T20:44:41.676 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.674+0000 7fe8bed9d640 1 --2- 172.21.15.195:0/472713801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0141730 0x7fe8c0147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:44:41.678 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8be59c640 1 --2- 172.21.15.195:0/472713801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c006c300 0x7fe8c00b72a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009870 tx=0x7fe8b001c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79e934ca936ea754 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:41.679 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8bdd9b640 1 -- 172.21.15.195:0/472713801 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00ab600 con 0x7fe8c006c300 2024-03-16T20:44:41.679 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8bdd9b640 1 -- 172.21.15.195:0/472713801 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b00a7050 con 0x7fe8c006c300 2024-03-16T20:44:41.679 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8bdd9b640 1 -- 172.21.15.195:0/472713801 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b00ab980 con 0x7fe8c006c300 2024-03-16T20:44:41.679 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 -- 172.21.15.195:0/472713801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c006c300 msgr2=0x7fe8c00b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.680 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 --2- 172.21.15.195:0/472713801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c006c300 0x7fe8c00b72a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe8b0009870 tx=0x7fe8b001c400 comp rx=0 tx=0).stop 2024-03-16T20:44:41.680 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 -- 172.21.15.195:0/472713801 shutdown_connections 2024-03-16T20:44:41.680 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 --2- 172.21.15.195:0/472713801 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c0141730 0x7fe8c0147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.680 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 --2- 172.21.15.195:0/472713801 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c006c300 0x7fe8c00b72a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.680 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 -- 172.21.15.195:0/472713801 >> 172.21.15.195:0/472713801 conn(0x7fe8c006fc00 msgr2=0x7fe8c0070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:41.680 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 -- 172.21.15.195:0/472713801 shutdown_connections 2024-03-16T20:44:41.680 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 -- 172.21.15.195:0/472713801 wait complete. 2024-03-16T20:44:41.682 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 Processor -- start 2024-03-16T20:44:41.682 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 -- start start 2024-03-16T20:44:41.682 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c006c300 0x7fe8c0134e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.682 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8c0141730 0x7fe8c01483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.683 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c01479a0 0x7fe8c01e2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.683 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c01356f0 con 0x7fe8c0141730 2024-03-16T20:44:41.683 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c0136a90 con 0x7fe8c006c300 2024-03-16T20:44:41.683 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c01378e0 con 0x7fe8c01479a0 2024-03-16T20:44:41.683 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8bed9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c01479a0 0x7fe8c01e2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.683 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8be59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c006c300 0x7fe8c0134e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.683 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8bed9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c01479a0 0x7fe8c01e2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:57696/0 (socket says 172.21.15.195:57696) 2024-03-16T20:44:41.683 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8be59c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c006c300 0x7fe8c0134e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:39474/0 (socket says 172.21.15.195:39474) 2024-03-16T20:44:41.683 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8bed9d640 1 -- 172.21.15.195:0/236643942 learned_addr learned my addr 172.21.15.195:0/236643942 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-16T20:44:41.683 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8b7fff640 1 --2- 172.21.15.195:0/236643942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8c0141730 0x7fe8c01483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.683 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8bed9d640 1 -- 172.21.15.195:0/236643942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c006c300 msgr2=0x7fe8c0134e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.683 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8bed9d640 1 --2- 172.21.15.195:0/236643942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c006c300 0x7fe8c0134e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.683 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8bed9d640 1 -- 172.21.15.195:0/236643942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8c0141730 msgr2=0x7fe8c01483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.684 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8bed9d640 1 --2- 172.21.15.195:0/236643942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8c0141730 0x7fe8c01483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.684 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8bed9d640 1 -- 172.21.15.195:0/236643942 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b8009a70 con 0x7fe8c01479a0 2024-03-16T20:44:41.684 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8bed9d640 1 --2- 172.21.15.195:0/236643942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c01479a0 0x7fe8c01e2800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009f00 tx=0x7fe8b800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:41.684 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8b6ffd640 1 -- 172.21.15.195:0/236643942 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8009d70 con 0x7fe8c01479a0 2024-03-16T20:44:41.684 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8b6ffd640 1 -- 172.21.15.195:0/236643942 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b800f040 con 0x7fe8c01479a0 2024-03-16T20:44:41.684 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 -- 172.21.15.195:0/236643942 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b00094d0 con 0x7fe8c01479a0 2024-03-16T20:44:41.684 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8b6ffd640 1 -- 172.21.15.195:0/236643942 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8003bc0 con 0x7fe8c01479a0 2024-03-16T20:44:41.684 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:41.678+0000 7ff351d72180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T20:44:41.685 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8c4886640 1 -- 172.21.15.195:0/236643942 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c014b670 con 0x7fe8c01479a0 2024-03-16T20:44:41.685 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8b6ffd640 1 -- 172.21.15.195:0/236643942 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fe8b8014480 con 0x7fe8c01479a0 2024-03-16T20:44:41.685 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.678+0000 7fe8b6ffd640 1 -- 172.21.15.195:0/236643942 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fe8b801dd50 con 0x7fe8c01479a0 2024-03-16T20:44:41.685 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.682+0000 7fe8c4886640 1 -- 172.21.15.195:0/236643942 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c0137b00 con 0x7fe8c01479a0 2024-03-16T20:44:41.685 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.682+0000 7fe8b6ffd640 1 -- 172.21.15.195:0/236643942 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fe8b8018030 con 0x7fe8c01479a0 2024-03-16T20:44:41.706 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:41.703+0000 7fe73285d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-16T20:44:41.735 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:41.734+0000 7ff351d72180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T20:44:41.748 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.746+0000 7fe8c4886640 1 -- 172.21.15.195:0/236643942 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3587f37a-c824-4dea-a154-297942ff1e1f", "id": 3} v 0) v1 -- 0x7fe8c0135860 con 0x7fe8c01479a0 2024-03-16T20:44:41.753 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.750+0000 7fe8b6ffd640 1 -- 172.21.15.195:0/236643942 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3587f37a-c824-4dea-a154-297942ff1e1f", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe8b801d8c0 con 0x7fe8c01479a0 2024-03-16T20:44:41.753 INFO:teuthology.orchestra.run.smithi195.stdout:3 2024-03-16T20:44:41.754 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.750+0000 7fe8c4886640 1 -- 172.21.15.195:0/236643942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c01479a0 msgr2=0x7fe8c01e2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.750+0000 7fe8c4886640 1 --2- 172.21.15.195:0/236643942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c01479a0 0x7fe8c01e2800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009f00 tx=0x7fe8b800c490 comp rx=0 tx=0).stop 2024-03-16T20:44:41.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.750+0000 7fe8c4886640 1 -- 172.21.15.195:0/236643942 shutdown_connections 2024-03-16T20:44:41.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.750+0000 7fe8c4886640 1 --2- 172.21.15.195:0/236643942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe8c01479a0 0x7fe8c01e2800 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.750+0000 7fe8c4886640 1 --2- 172.21.15.195:0/236643942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe8c0141730 0x7fe8c01483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.750+0000 7fe8c4886640 1 --2- 172.21.15.195:0/236643942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe8c006c300 0x7fe8c0134e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.750+0000 7fe8c4886640 1 -- 172.21.15.195:0/236643942 >> 172.21.15.195:0/236643942 conn(0x7fe8c006fc00 msgr2=0x7fe8c0133640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:41.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.754+0000 7fe8c4886640 1 -- 172.21.15.195:0/236643942 shutdown_connections 2024-03-16T20:44:41.755 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.754+0000 7fe8c4886640 1 -- 172.21.15.195:0/236643942 wait complete. 2024-03-16T20:44:41.757 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:41.755+0000 7fe73285d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-16T20:44:41.769 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new d7fa79af-de38-4c3a-96e6-e97126092629 4 2024-03-16T20:44:41.816 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:41.814+0000 7ff351d72180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T20:44:41.834 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:41.831+0000 7fe73285d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-16T20:44:41.871 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:41.870+0000 7ff351d72180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T20:44:41.887 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:41.883+0000 7fe73285d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-16T20:44:41.921 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.918+0000 7f87456e4640 1 Processor -- start 2024-03-16T20:44:41.921 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.918+0000 7f87456e4640 1 -- start start 2024-03-16T20:44:41.922 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.918+0000 7f87456e4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f87400b7d40 0x7f87400b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.922 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.918+0000 7f87456e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8740148480 0x7f874014f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.922 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.918+0000 7f87456e4640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87400b6740 con 0x7f87400b7540 2024-03-16T20:44:41.922 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.918+0000 7f87456e4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87400b23d0 con 0x7f87400b7d40 2024-03-16T20:44:41.922 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.918+0000 7f87456e4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87400b7110 con 0x7f8740148480 2024-03-16T20:44:41.922 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.918+0000 7f873e7fc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f87400b7d40 0x7f87400b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.918+0000 7f873f7fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8740148480 0x7f874014f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.918+0000 7f873e7fc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f87400b7d40 0x7f87400b6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:39490/0 (socket says 172.21.15.195:39490) 2024-03-16T20:44:41.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.918+0000 7f873f7fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8740148480 0x7f874014f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:57700/0 (socket says 172.21.15.195:57700) 2024-03-16T20:44:41.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.918+0000 7f873e7fc640 1 -- 172.21.15.195:0/1770267247 learned_addr learned my addr 172.21.15.195:0/1770267247 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-16T20:44:41.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.918+0000 7f873e7fc640 1 -- 172.21.15.195:0/1770267247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8740148480 msgr2=0x7f874014f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:41.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.918+0000 7f873e7fc640 1 --2- 172.21.15.195:0/1770267247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8740148480 0x7f874014f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.918+0000 7f873e7fc640 1 -- 172.21.15.195:0/1770267247 >> v1:172.21.15.122:6789/0 conn(0x7f87400b7540 legacy=0x7f874013ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:41.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.918+0000 7f873e7fc640 1 -- 172.21.15.195:0/1770267247 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87400b7920 con 0x7f87400b7d40 2024-03-16T20:44:41.923 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.918+0000 7f873f7fe640 1 --2- 172.21.15.195:0/1770267247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8740148480 0x7f874014f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:44:41.924 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.922+0000 7f873e7fc640 1 --2- 172.21.15.195:0/1770267247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87400b7d40 0x7f87400b6bd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f87340098a0 tx=0x7f873401c440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15afef314c41d0b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:41.924 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.922+0000 7f873dffb640 1 -- 172.21.15.195:0/1770267247 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87340ab600 con 0x7f87400b7d40 2024-03-16T20:44:41.924 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.922+0000 7f873dffb640 1 -- 172.21.15.195:0/1770267247 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87340ab760 con 0x7f87400b7d40 2024-03-16T20:44:41.924 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.922+0000 7f873dffb640 1 -- 172.21.15.195:0/1770267247 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87340abaf0 con 0x7f87400b7d40 2024-03-16T20:44:41.925 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.922+0000 7f87456e4640 1 -- 172.21.15.195:0/1770267247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87400b7d40 msgr2=0x7f87400b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.925 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.922+0000 7f87456e4640 1 --2- 172.21.15.195:0/1770267247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87400b7d40 0x7f87400b6bd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f87340098a0 tx=0x7f873401c440 comp rx=0 tx=0).stop 2024-03-16T20:44:41.925 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.922+0000 7f87456e4640 1 -- 172.21.15.195:0/1770267247 shutdown_connections 2024-03-16T20:44:41.925 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.922+0000 7f87456e4640 1 --2- 172.21.15.195:0/1770267247 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8740148480 0x7f874014f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.925 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.922+0000 7f87456e4640 1 --2- 172.21.15.195:0/1770267247 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87400b7d40 0x7f87400b6bd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.926 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.922+0000 7f87456e4640 1 -- 172.21.15.195:0/1770267247 >> 172.21.15.195:0/1770267247 conn(0x7f874006fc00 msgr2=0x7f8740070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:41.926 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.922+0000 7f87456e4640 1 -- 172.21.15.195:0/1770267247 shutdown_connections 2024-03-16T20:44:41.926 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.922+0000 7f87456e4640 1 -- 172.21.15.195:0/1770267247 wait complete. 2024-03-16T20:44:41.928 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.922+0000 7f87456e4640 1 Processor -- start 2024-03-16T20:44:41.928 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.922+0000 7f87456e4640 1 -- start start 2024-03-16T20:44:41.928 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f87456e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87400b7540 0x7f874006db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.928 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f87456e4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87400b7d40 0x7f874013c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.928 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f87456e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8740148480 0x7f874013c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:41.928 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f87456e4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874014f550 con 0x7f8740148480 2024-03-16T20:44:41.928 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f87456e4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874013bf60 con 0x7f87400b7d40 2024-03-16T20:44:41.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f87456e4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874013cd90 con 0x7f87400b7540 2024-03-16T20:44:41.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f873effd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87400b7540 0x7f874006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f873e7fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87400b7d40 0x7f874013c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f873effd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87400b7540 0x7f874006db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:57716/0 (socket says 172.21.15.195:57716) 2024-03-16T20:44:41.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f873e7fc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87400b7d40 0x7f874013c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:39506/0 (socket says 172.21.15.195:39506) 2024-03-16T20:44:41.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f873effd640 1 -- 172.21.15.195:0/491969015 learned_addr learned my addr 172.21.15.195:0/491969015 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-16T20:44:41.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f873f7fe640 1 --2- 172.21.15.195:0/491969015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8740148480 0x7f874013c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:41.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f873e7fc640 1 -- 172.21.15.195:0/491969015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87400b7540 msgr2=0x7f874006db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f873e7fc640 1 --2- 172.21.15.195:0/491969015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87400b7540 0x7f874006db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f873e7fc640 1 -- 172.21.15.195:0/491969015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8740148480 msgr2=0x7f874013c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:41.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f873e7fc640 1 --2- 172.21.15.195:0/491969015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8740148480 0x7f874013c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:41.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f873e7fc640 1 -- 172.21.15.195:0/491969015 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8738009a70 con 0x7f87400b7d40 2024-03-16T20:44:41.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f873effd640 1 --2- 172.21.15.195:0/491969015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87400b7540 0x7f874006db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:44:41.929 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f873e7fc640 1 --2- 172.21.15.195:0/491969015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87400b7d40 0x7f874013c170 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f87340ac000 tx=0x7f873401ac80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:41.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f87277fe640 1 -- 172.21.15.195:0/491969015 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87340b4800 con 0x7f87400b7d40 2024-03-16T20:44:41.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f87277fe640 1 -- 172.21.15.195:0/491969015 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87340b4980 con 0x7f87400b7d40 2024-03-16T20:44:41.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f87277fe640 1 -- 172.21.15.195:0/491969015 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87340b4d30 con 0x7f87400b7d40 2024-03-16T20:44:41.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f87456e4640 1 -- 172.21.15.195:0/491969015 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87340094d0 con 0x7f87400b7d40 2024-03-16T20:44:41.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f87456e4640 1 -- 172.21.15.195:0/491969015 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87401df780 con 0x7f87400b7d40 2024-03-16T20:44:41.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f87277fe640 1 -- 172.21.15.195:0/491969015 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f87340c7440 con 0x7f87400b7d40 2024-03-16T20:44:41.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f87277fe640 1 -- 172.21.15.195:0/491969015 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f87340be900 con 0x7f87400b7d40 2024-03-16T20:44:41.930 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.926+0000 7f87456e4640 1 -- 172.21.15.195:0/491969015 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87400b2db0 con 0x7f87400b7d40 2024-03-16T20:44:41.931 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.930+0000 7f87277fe640 1 -- 172.21.15.195:0/491969015 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f87340beba0 con 0x7f87400b7d40 2024-03-16T20:44:41.998 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:41.994+0000 7f87456e4640 1 -- 172.21.15.195:0/491969015 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d7fa79af-de38-4c3a-96e6-e97126092629", "id": 4} v 0) v1 -- 0x7f87400b18d0 con 0x7f87400b7d40 2024-03-16T20:44:42.005 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.002+0000 7f87277fe640 1 -- 172.21.15.195:0/491969015 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d7fa79af-de38-4c3a-96e6-e97126092629", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f87340bee10 con 0x7f87400b7d40 2024-03-16T20:44:42.005 INFO:teuthology.orchestra.run.smithi195.stdout:4 2024-03-16T20:44:42.006 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.002+0000 7f87456e4640 1 -- 172.21.15.195:0/491969015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87400b7d40 msgr2=0x7f874013c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.002+0000 7f87456e4640 1 --2- 172.21.15.195:0/491969015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87400b7d40 0x7f874013c170 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f87340ac000 tx=0x7f873401ac80 comp rx=0 tx=0).stop 2024-03-16T20:44:42.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.002+0000 7f87456e4640 1 -- 172.21.15.195:0/491969015 shutdown_connections 2024-03-16T20:44:42.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.002+0000 7f87456e4640 1 --2- 172.21.15.195:0/491969015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8740148480 0x7f874013c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.002+0000 7f87456e4640 1 --2- 172.21.15.195:0/491969015 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f87400b7d40 0x7f874013c170 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.002+0000 7f87456e4640 1 --2- 172.21.15.195:0/491969015 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f87400b7540 0x7f874006db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.002+0000 7f87456e4640 1 -- 172.21.15.195:0/491969015 >> 172.21.15.195:0/491969015 conn(0x7f874006fc00 msgr2=0x7f87400b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:42.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.002+0000 7f87456e4640 1 -- 172.21.15.195:0/491969015 shutdown_connections 2024-03-16T20:44:42.007 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.002+0000 7f87456e4640 1 -- 172.21.15.195:0/491969015 wait complete. 2024-03-16T20:44:42.023 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new f5d2b51c-29b9-40be-bcb7-134c4e1343b2 5 2024-03-16T20:44:42.027 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:42.026+0000 7ff351d72180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T20:44:42.035 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:42.031+0000 7fe73285d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-16T20:44:42.092 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:42.090+0000 7ff351d72180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T20:44:42.097 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:42.095+0000 7fe73285d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-16T20:44:42.132 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb5a690640 1 Processor -- start 2024-03-16T20:44:42.133 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb5a690640 1 -- start start 2024-03-16T20:44:42.135 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb5a690640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fbb540b90c0 0x7fbb5406ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.135 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb5a690640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5406daf0 con 0x7fbb54144c00 2024-03-16T20:44:42.135 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb5a690640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb540b1390 con 0x7fbb54144290 2024-03-16T20:44:42.136 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb5a690640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb5406dd40 con 0x7fbb540b90c0 2024-03-16T20:44:42.136 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb53fff640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fbb54144290 0x7fbb5413df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.195:59510/0 (socket says 172.21.15.195:59510) 2024-03-16T20:44:42.136 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb53fff640 1 -- 172.21.15.195:0/2867478217 learned_addr learned my addr 172.21.15.195:0/2867478217 (peer_addr_for_me v1:172.21.15.195:0/0) 2024-03-16T20:44:42.136 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb537fe640 1 --2- 172.21.15.195:0/2867478217 >> v2:172.21.15.122:3300/0 conn(0x7fbb540b90c0 0x7fbb5406ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.136 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb52ffd640 1 -- 172.21.15.195:0/2867478217 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913576436 0 0) 0x7fbb540b1390 con 0x7fbb54144290 2024-03-16T20:44:42.136 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb52ffd640 1 -- 172.21.15.195:0/2867478217 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb3c003530 con 0x7fbb54144290 2024-03-16T20:44:42.136 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb52ffd640 1 -- 172.21.15.195:0/2867478217 <== mon.1 v1:172.21.15.195:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3168034941 0 0) 0x7fbb3c003530 con 0x7fbb54144290 2024-03-16T20:44:42.136 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb52ffd640 1 -- 172.21.15.195:0/2867478217 >> v2:172.21.15.122:3300/0 conn(0x7fbb540b90c0 msgr2=0x7fbb5406ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:42.136 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb52ffd640 1 --2- 172.21.15.195:0/2867478217 >> v2:172.21.15.122:3300/0 conn(0x7fbb540b90c0 0x7fbb5406ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.136 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb52ffd640 1 -- 172.21.15.195:0/2867478217 >> v1:172.21.15.122:6789/0 conn(0x7fbb54144c00 legacy=0x7fbb5414e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.136 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb52ffd640 1 -- 172.21.15.195:0/2867478217 --> v1:172.21.15.195:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5413e690 con 0x7fbb54144290 2024-03-16T20:44:42.136 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb52ffd640 1 -- 172.21.15.195:0/2867478217 <== mon.1 v1:172.21.15.195:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7fbb44001680 con 0x7fbb54144290 2024-03-16T20:44:42.136 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb52ffd640 1 -- 172.21.15.195:0/2867478217 >> v1:172.21.15.195:6789/0 conn(0x7fbb54144290 legacy=0x7fbb5413df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.137 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb52ffd640 1 --2- 172.21.15.195:0/2867478217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb3c003980 0x7fbb3c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.137 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb52ffd640 1 --2- 172.21.15.195:0/2867478217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb3c0047d0 0x7fbb3c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.137 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb52ffd640 1 --2- 172.21.15.195:0/2867478217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb3c005390 0x7fbb3c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.137 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb52ffd640 1 -- 172.21.15.195:0/2867478217 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5406dd40 con 0x7fbb3c005390 2024-03-16T20:44:42.137 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb52ffd640 1 -- 172.21.15.195:0/2867478217 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c00c220 con 0x7fbb3c0047d0 2024-03-16T20:44:42.137 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb52ffd640 1 -- 172.21.15.195:0/2867478217 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c00c3e0 con 0x7fbb3c003980 2024-03-16T20:44:42.137 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb53fff640 1 --2- 172.21.15.195:0/2867478217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb3c003980 0x7fbb3c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.137 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.130+0000 7fbb537fe640 1 --2- 172.21.15.195:0/2867478217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb3c0047d0 0x7fbb3c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.137 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb537fe640 1 -- 172.21.15.195:0/2867478217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb3c003980 msgr2=0x7fbb3c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:42.137 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb537fe640 1 --2- 172.21.15.195:0/2867478217 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb3c003980 0x7fbb3c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.137 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb537fe640 1 -- 172.21.15.195:0/2867478217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb3c005390 msgr2=0x7fbb3c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:42.137 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb58c2d640 1 --2- 172.21.15.195:0/2867478217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb3c005390 0x7fbb3c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.137 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb537fe640 1 --2- 172.21.15.195:0/2867478217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb3c005390 0x7fbb3c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.137 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb537fe640 1 -- 172.21.15.195:0/2867478217 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c003530 con 0x7fbb3c0047d0 2024-03-16T20:44:42.138 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb58c2d640 1 --2- 172.21.15.195:0/2867478217 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb3c005390 0x7fbb3c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:44:42.138 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb58c2d640 1 -- 172.21.15.195:0/2867478217 reap_dead start 2024-03-16T20:44:42.139 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb537fe640 1 --2- 172.21.15.195:0/2867478217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb3c0047d0 0x7fbb3c004c50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800ba10 tx=0x7fbb4801c680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5eecb981935c8d8c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:42.139 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb52ffd640 1 -- 172.21.15.195:0/2867478217 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb480ac620 con 0x7fbb3c0047d0 2024-03-16T20:44:42.139 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb52ffd640 1 -- 172.21.15.195:0/2867478217 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb480a6040 con 0x7fbb3c0047d0 2024-03-16T20:44:42.139 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb52ffd640 1 -- 172.21.15.195:0/2867478217 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb480ac9c0 con 0x7fbb3c0047d0 2024-03-16T20:44:42.139 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb5a690640 1 -- 172.21.15.195:0/2867478217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb3c0047d0 msgr2=0x7fbb3c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.140 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb5a690640 1 --2- 172.21.15.195:0/2867478217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb3c0047d0 0x7fbb3c004c50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800ba10 tx=0x7fbb4801c680 comp rx=0 tx=0).stop 2024-03-16T20:44:42.140 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb5a690640 1 -- 172.21.15.195:0/2867478217 shutdown_connections 2024-03-16T20:44:42.140 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb5a690640 1 --2- 172.21.15.195:0/2867478217 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb3c0047d0 0x7fbb3c004c50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.140 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb5a690640 1 -- 172.21.15.195:0/2867478217 >> 172.21.15.195:0/2867478217 conn(0x7fbb54070d30 msgr2=0x7fbb54071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:42.140 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb5a690640 1 -- 172.21.15.195:0/2867478217 shutdown_connections 2024-03-16T20:44:42.140 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.134+0000 7fbb5a690640 1 -- 172.21.15.195:0/2867478217 wait complete. 2024-03-16T20:44:42.140 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb5a690640 1 Processor -- start 2024-03-16T20:44:42.140 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb5a690640 1 -- start start 2024-03-16T20:44:42.142 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb5a690640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb3c0047d0 0x7fbb54146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.142 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb5a690640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb541dd370 0x7fbb541dd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.142 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb5a690640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb541e0260 0x7fbb541e06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.143 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb5a690640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5413d2c0 con 0x7fbb3c0047d0 2024-03-16T20:44:42.143 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb5a690640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb54146a90 con 0x7fbb541e0260 2024-03-16T20:44:42.143 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb5a690640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5413d7b0 con 0x7fbb541dd370 2024-03-16T20:44:42.143 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb537fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb541dd370 0x7fbb541dd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.143 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb537fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb541dd370 0x7fbb541dd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:57732/0 (socket says 172.21.15.195:57732) 2024-03-16T20:44:42.143 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb537fe640 1 -- 172.21.15.195:0/45318598 learned_addr learned my addr 172.21.15.195:0/45318598 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-16T20:44:42.143 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb58c2d640 1 --2- 172.21.15.195:0/45318598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb541e0260 0x7fbb541e06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.143 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb53fff640 1 --2- 172.21.15.195:0/45318598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb3c0047d0 0x7fbb54146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.143 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb537fe640 1 -- 172.21.15.195:0/45318598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb541e0260 msgr2=0x7fbb541e06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.143 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb537fe640 1 --2- 172.21.15.195:0/45318598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb541e0260 0x7fbb541e06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.143 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb537fe640 1 -- 172.21.15.195:0/45318598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb3c0047d0 msgr2=0x7fbb54146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.143 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb537fe640 1 --2- 172.21.15.195:0/45318598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb3c0047d0 0x7fbb54146550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.143 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb537fe640 1 -- 172.21.15.195:0/45318598 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb541dce90 con 0x7fbb541dd370 2024-03-16T20:44:42.143 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb58c2d640 1 --2- 172.21.15.195:0/45318598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb541e0260 0x7fbb541e06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:44:42.144 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb537fe640 1 --2- 172.21.15.195:0/45318598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb541dd370 0x7fbb541dd7f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbb4801cb60 tx=0x7fbb3c005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:42.144 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb50ff9640 1 -- 172.21.15.195:0/45318598 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb480acc40 con 0x7fbb541dd370 2024-03-16T20:44:42.144 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb50ff9640 1 -- 172.21.15.195:0/45318598 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb480b9070 con 0x7fbb541dd370 2024-03-16T20:44:42.144 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb5a690640 1 -- 172.21.15.195:0/45318598 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb5413ea40 con 0x7fbb541dd370 2024-03-16T20:44:42.144 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb50ff9640 1 -- 172.21.15.195:0/45318598 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb480bf400 con 0x7fbb541dd370 2024-03-16T20:44:42.144 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb5a690640 1 -- 172.21.15.195:0/45318598 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb54147150 con 0x7fbb541dd370 2024-03-16T20:44:42.144 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb50ff9640 1 -- 172.21.15.195:0/45318598 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbb480bf8f0 con 0x7fbb541dd370 2024-03-16T20:44:42.144 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb50ff9640 1 -- 172.21.15.195:0/45318598 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fbb480b5970 con 0x7fbb541dd370 2024-03-16T20:44:42.144 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.138+0000 7fbb5a690640 1 -- 172.21.15.195:0/45318598 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb541dccf0 con 0x7fbb541dd370 2024-03-16T20:44:42.144 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.142+0000 7fbb50ff9640 1 -- 172.21.15.195:0/45318598 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbb541dccf0 con 0x7fbb541dd370 2024-03-16T20:44:42.208 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.206+0000 7fbb5a690640 1 -- 172.21.15.195:0/45318598 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f5d2b51c-29b9-40be-bcb7-134c4e1343b2", "id": 5} v 0) v1 -- 0x7fbb540ba6b0 con 0x7fbb541dd370 2024-03-16T20:44:42.213 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.210+0000 7fbb50ff9640 1 -- 172.21.15.195:0/45318598 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f5d2b51c-29b9-40be-bcb7-134c4e1343b2", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbb480bb070 con 0x7fbb541dd370 2024-03-16T20:44:42.214 INFO:teuthology.orchestra.run.smithi195.stdout:5 2024-03-16T20:44:42.215 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.214+0000 7fbb5a690640 1 -- 172.21.15.195:0/45318598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb541dd370 msgr2=0x7fbb541dd7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.215 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.214+0000 7fbb5a690640 1 --2- 172.21.15.195:0/45318598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb541dd370 0x7fbb541dd7f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbb4801cb60 tx=0x7fbb3c005800 comp rx=0 tx=0).stop 2024-03-16T20:44:42.215 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.214+0000 7fbb5a690640 1 -- 172.21.15.195:0/45318598 shutdown_connections 2024-03-16T20:44:42.215 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.214+0000 7fbb5a690640 1 --2- 172.21.15.195:0/45318598 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbb541e0260 0x7fbb541e06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.215 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.214+0000 7fbb5a690640 1 --2- 172.21.15.195:0/45318598 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbb541dd370 0x7fbb541dd7f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.215 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.214+0000 7fbb5a690640 1 --2- 172.21.15.195:0/45318598 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbb3c0047d0 0x7fbb54146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.215 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.214+0000 7fbb5a690640 1 -- 172.21.15.195:0/45318598 >> 172.21.15.195:0/45318598 conn(0x7fbb54070d30 msgr2=0x7fbb54138de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:42.215 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.214+0000 7fbb5a690640 1 -- 172.21.15.195:0/45318598 shutdown_connections 2024-03-16T20:44:42.215 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.214+0000 7fbb5a690640 1 -- 172.21.15.195:0/45318598 wait complete. 2024-03-16T20:44:42.230 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new 4d70db10-084a-4b65-b9b9-759620154490 6 2024-03-16T20:44:42.338 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.334+0000 7f9db606d640 1 Processor -- start 2024-03-16T20:44:42.339 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.334+0000 7f9db606d640 1 -- start start 2024-03-16T20:44:42.339 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.334+0000 7f9db606d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9db009b0a0 0x7f9db009b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.339 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.334+0000 7f9db606d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db01442a0 0x7f9db0144730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.339 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.334+0000 7f9db606d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db009a220 con 0x7f9db0066fa0 2024-03-16T20:44:42.339 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.334+0000 7f9db606d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db009bdf0 con 0x7f9db009b0a0 2024-03-16T20:44:42.339 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.334+0000 7f9db606d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db00af730 con 0x7f9db01442a0 2024-03-16T20:44:42.339 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.334+0000 7f9daffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db01442a0 0x7f9db0144730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.339 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.334+0000 7f9daffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db01442a0 0x7f9db0144730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.195:57736/0 (socket says 172.21.15.195:57736) 2024-03-16T20:44:42.339 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.334+0000 7f9daffff640 1 -- 172.21.15.195:0/1080736007 learned_addr learned my addr 172.21.15.195:0/1080736007 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-16T20:44:42.339 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.334+0000 7f9daeffd640 1 --2- 172.21.15.195:0/1080736007 >> v2:172.21.15.195:3300/0 conn(0x7f9db009b0a0 0x7f9db009b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.339 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.334+0000 7f9daffff640 1 -- 172.21.15.195:0/1080736007 >> v2:172.21.15.195:3300/0 conn(0x7f9db009b0a0 msgr2=0x7f9db009b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:42.339 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.334+0000 7f9daffff640 1 --2- 172.21.15.195:0/1080736007 >> v2:172.21.15.195:3300/0 conn(0x7f9db009b0a0 0x7f9db009b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.340 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.334+0000 7f9daffff640 1 -- 172.21.15.195:0/1080736007 >> v1:172.21.15.122:6789/0 conn(0x7f9db0066fa0 legacy=0x7f9db009a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:42.340 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.334+0000 7f9daffff640 1 -- 172.21.15.195:0/1080736007 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db0067380 con 0x7f9db01442a0 2024-03-16T20:44:42.343 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9daffff640 1 --2- 172.21.15.195:0/1080736007 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db01442a0 0x7f9db0144730 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009e50 tx=0x7f9da801c930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e98307a58cd6601 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:42.343 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9dae7fc640 1 -- 172.21.15.195:0/1080736007 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da80a57d0 con 0x7f9db01442a0 2024-03-16T20:44:42.343 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9dae7fc640 1 -- 172.21.15.195:0/1080736007 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da80a5930 con 0x7f9db01442a0 2024-03-16T20:44:42.343 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9dae7fc640 1 -- 172.21.15.195:0/1080736007 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da80a5cc0 con 0x7f9db01442a0 2024-03-16T20:44:42.343 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9db606d640 1 -- 172.21.15.195:0/1080736007 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db01442a0 msgr2=0x7f9db0144730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.343 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9db606d640 1 --2- 172.21.15.195:0/1080736007 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db01442a0 0x7f9db0144730 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009e50 tx=0x7f9da801c930 comp rx=0 tx=0).stop 2024-03-16T20:44:42.344 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9db606d640 1 -- 172.21.15.195:0/1080736007 shutdown_connections 2024-03-16T20:44:42.344 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9db606d640 1 --2- 172.21.15.195:0/1080736007 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db01442a0 0x7f9db0144730 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.344 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9db606d640 1 --2- 172.21.15.195:0/1080736007 >> v2:172.21.15.195:3300/0 conn(0x7f9db009b0a0 0x7f9db009b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.344 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9db606d640 1 -- 172.21.15.195:0/1080736007 >> 172.21.15.195:0/1080736007 conn(0x7f9db006d2b0 msgr2=0x7f9db006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:42.344 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9db606d640 1 -- 172.21.15.195:0/1080736007 shutdown_connections 2024-03-16T20:44:42.344 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9db606d640 1 -- 172.21.15.195:0/1080736007 wait complete. 2024-03-16T20:44:42.344 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9db606d640 1 Processor -- start 2024-03-16T20:44:42.344 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9db606d640 1 -- start start 2024-03-16T20:44:42.344 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9db606d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db0066fa0 0x7f9db00b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.344 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9db606d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9db009b0a0 0x7f9db00b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.344 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9db606d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9db01442a0 0x7f9db00b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.344 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9db606d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db00b0e10 con 0x7f9db01442a0 2024-03-16T20:44:42.344 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9db606d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db0150e90 con 0x7f9db009b0a0 2024-03-16T20:44:42.345 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9db606d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db00b1890 con 0x7f9db0066fa0 2024-03-16T20:44:42.345 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9daeffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9db009b0a0 0x7f9db00b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.345 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9daf7fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db0066fa0 0x7f9db00b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.345 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9daeffd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9db009b0a0 0x7f9db00b1350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:39534/0 (socket says 172.21.15.195:39534) 2024-03-16T20:44:42.345 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9daeffd640 1 -- 172.21.15.195:0/426348904 learned_addr learned my addr 172.21.15.195:0/426348904 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-16T20:44:42.345 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9daffff640 1 --2- 172.21.15.195:0/426348904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9db01442a0 0x7f9db00b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.345 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9daeffd640 1 -- 172.21.15.195:0/426348904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db0066fa0 msgr2=0x7f9db00b08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.345 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9daeffd640 1 --2- 172.21.15.195:0/426348904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db0066fa0 0x7f9db00b08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.345 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9daeffd640 1 -- 172.21.15.195:0/426348904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9db01442a0 msgr2=0x7f9db00b2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.345 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9daeffd640 1 --2- 172.21.15.195:0/426348904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9db01442a0 0x7f9db00b2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.345 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9daeffd640 1 -- 172.21.15.195:0/426348904 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da8009b30 con 0x7f9db009b0a0 2024-03-16T20:44:42.345 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9daf7fe640 1 --2- 172.21.15.195:0/426348904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db0066fa0 0x7f9db00b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:44:42.345 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9daeffd640 1 --2- 172.21.15.195:0/426348904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9db009b0a0 0x7f9db00b1350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9da400e960 tx=0x7f9da400ee20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:42.345 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9d97fff640 1 -- 172.21.15.195:0/426348904 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4009690 con 0x7f9db009b0a0 2024-03-16T20:44:42.346 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9d97fff640 1 -- 172.21.15.195:0/426348904 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da400ca00 con 0x7f9db009b0a0 2024-03-16T20:44:42.346 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9db606d640 1 -- 172.21.15.195:0/426348904 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db00b1af0 con 0x7f9db009b0a0 2024-03-16T20:44:42.346 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.338+0000 7f9d97fff640 1 -- 172.21.15.195:0/426348904 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da400cd70 con 0x7f9db009b0a0 2024-03-16T20:44:42.346 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.342+0000 7f9db606d640 1 -- 172.21.15.195:0/426348904 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db0151490 con 0x7f9db009b0a0 2024-03-16T20:44:42.346 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.342+0000 7f9d97fff640 1 -- 172.21.15.195:0/426348904 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9da40059a0 con 0x7f9db009b0a0 2024-03-16T20:44:42.346 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.342+0000 7f9d97fff640 1 -- 172.21.15.195:0/426348904 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f9da4013a60 con 0x7f9db009b0a0 2024-03-16T20:44:42.346 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.342+0000 7f9db606d640 1 -- 172.21.15.195:0/426348904 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db0074bb0 con 0x7f9db009b0a0 2024-03-16T20:44:42.346 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.342+0000 7f9d97fff640 1 -- 172.21.15.195:0/426348904 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9da4021060 con 0x7f9db009b0a0 2024-03-16T20:44:42.412 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.410+0000 7f9db606d640 1 -- 172.21.15.195:0/426348904 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4d70db10-084a-4b65-b9b9-759620154490", "id": 6} v 0) v1 -- 0x7f9db00b0020 con 0x7f9db009b0a0 2024-03-16T20:44:42.418 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.414+0000 7f9d97fff640 1 -- 172.21.15.195:0/426348904 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4d70db10-084a-4b65-b9b9-759620154490", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9da4017070 con 0x7f9db009b0a0 2024-03-16T20:44:42.418 INFO:teuthology.orchestra.run.smithi195.stdout:6 2024-03-16T20:44:42.420 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.418+0000 7f9db606d640 1 -- 172.21.15.195:0/426348904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9db009b0a0 msgr2=0x7f9db00b1350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.420 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.418+0000 7f9db606d640 1 --2- 172.21.15.195:0/426348904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9db009b0a0 0x7f9db00b1350 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9da400e960 tx=0x7f9da400ee20 comp rx=0 tx=0).stop 2024-03-16T20:44:42.420 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.418+0000 7f9db606d640 1 -- 172.21.15.195:0/426348904 shutdown_connections 2024-03-16T20:44:42.421 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.418+0000 7f9db606d640 1 --2- 172.21.15.195:0/426348904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9db01442a0 0x7f9db00b2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.421 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.418+0000 7f9db606d640 1 --2- 172.21.15.195:0/426348904 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9db009b0a0 0x7f9db00b1350 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.421 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.418+0000 7f9db606d640 1 --2- 172.21.15.195:0/426348904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9db0066fa0 0x7f9db00b08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.421 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.418+0000 7f9db606d640 1 -- 172.21.15.195:0/426348904 >> 172.21.15.195:0/426348904 conn(0x7f9db006d2b0 msgr2=0x7f9db0074320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:42.421 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.418+0000 7f9db606d640 1 -- 172.21.15.195:0/426348904 shutdown_connections 2024-03-16T20:44:42.421 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.418+0000 7f9db606d640 1 -- 172.21.15.195:0/426348904 wait complete. 2024-03-16T20:44:42.448 DEBUG:teuthology.orchestra.run.smithi195:> sudo ceph --cluster ceph osd new ac0af635-e34b-4876-a01d-d9c98833385e 7 2024-03-16T20:44:42.589 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.586+0000 7ffa35ebd640 1 Processor -- start 2024-03-16T20:44:42.589 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.586+0000 7ffa35ebd640 1 -- start start 2024-03-16T20:44:42.590 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.586+0000 7ffa35ebd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ffa300b7d40 0x7ffa300b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.591 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.586+0000 7ffa35ebd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ffa30148480 0x7ffa3014f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.591 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.586+0000 7ffa35ebd640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa300b6740 con 0x7ffa300b7540 2024-03-16T20:44:42.591 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.586+0000 7ffa35ebd640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa300b23d0 con 0x7ffa30148480 2024-03-16T20:44:42.591 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.586+0000 7ffa35ebd640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa300b7110 con 0x7ffa300b7d40 2024-03-16T20:44:42.591 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.586+0000 7ffa2effd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ffa300b7d40 0x7ffa300b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.591 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.586+0000 7ffa2f7fe640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7ffa300b7540 0x7ffa3013ef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.195:59522/0 (socket says 172.21.15.195:59522) 2024-03-16T20:44:42.591 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.586+0000 7ffa2f7fe640 1 -- 172.21.15.195:0/2577485944 learned_addr learned my addr 172.21.15.195:0/2577485944 (peer_addr_for_me v1:172.21.15.195:0/0) 2024-03-16T20:44:42.592 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.586+0000 7ffa2ffff640 1 --2- 172.21.15.195:0/2577485944 >> v2:172.21.15.122:3300/0 conn(0x7ffa30148480 0x7ffa3014f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.592 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.586+0000 7ffa2effd640 1 -- 172.21.15.195:0/2577485944 >> v2:172.21.15.122:3300/0 conn(0x7ffa30148480 msgr2=0x7ffa3014f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:42.592 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.586+0000 7ffa2effd640 1 --2- 172.21.15.195:0/2577485944 >> v2:172.21.15.122:3300/0 conn(0x7ffa30148480 0x7ffa3014f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.592 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.586+0000 7ffa2effd640 1 -- 172.21.15.195:0/2577485944 >> v1:172.21.15.195:6789/0 conn(0x7ffa300b7540 legacy=0x7ffa3013ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.592 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.586+0000 7ffa2e7fc640 1 -- 172.21.15.195:0/2577485944 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047759837 0 0) 0x7ffa300b6740 con 0x7ffa300b7540 2024-03-16T20:44:42.592 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.586+0000 7ffa2effd640 1 -- 172.21.15.195:0/2577485944 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa300b7920 con 0x7ffa300b7d40 2024-03-16T20:44:42.592 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.586+0000 7ffa2ffff640 1 --2- 172.21.15.195:0/2577485944 >> v2:172.21.15.122:3300/0 conn(0x7ffa30148480 0x7ffa3014f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:44:42.593 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.590+0000 7ffa2effd640 1 --2- 172.21.15.195:0/2577485944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa300b7d40 0x7ffa300b6bd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ffa300b2d80 tx=0x7ffa2401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=185ea129f7d36eca server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:42.593 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.590+0000 7ffa2e7fc640 1 -- 172.21.15.195:0/2577485944 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa240ac620 con 0x7ffa300b7d40 2024-03-16T20:44:42.593 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.590+0000 7ffa2e7fc640 1 -- 172.21.15.195:0/2577485944 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa240ac780 con 0x7ffa300b7d40 2024-03-16T20:44:42.593 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.590+0000 7ffa2e7fc640 1 -- 172.21.15.195:0/2577485944 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa240acb10 con 0x7ffa300b7d40 2024-03-16T20:44:42.593 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.590+0000 7ffa35ebd640 1 -- 172.21.15.195:0/2577485944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa300b7d40 msgr2=0x7ffa300b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.594 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.590+0000 7ffa35ebd640 1 --2- 172.21.15.195:0/2577485944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa300b7d40 0x7ffa300b6bd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ffa300b2d80 tx=0x7ffa2401c3c0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.594 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.590+0000 7ffa35ebd640 1 -- 172.21.15.195:0/2577485944 shutdown_connections 2024-03-16T20:44:42.594 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.590+0000 7ffa35ebd640 1 --2- 172.21.15.195:0/2577485944 >> v2:172.21.15.122:3300/0 conn(0x7ffa30148480 0x7ffa3014f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.594 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.590+0000 7ffa35ebd640 1 --2- 172.21.15.195:0/2577485944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa300b7d40 0x7ffa300b6bd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.594 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.590+0000 7ffa35ebd640 1 -- 172.21.15.195:0/2577485944 >> 172.21.15.195:0/2577485944 conn(0x7ffa3006fc00 msgr2=0x7ffa30070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:42.594 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.590+0000 7ffa35ebd640 1 -- 172.21.15.195:0/2577485944 shutdown_connections 2024-03-16T20:44:42.594 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.590+0000 7ffa35ebd640 1 -- 172.21.15.195:0/2577485944 wait complete. 2024-03-16T20:44:42.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.590+0000 7ffa35ebd640 1 Processor -- start 2024-03-16T20:44:42.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.590+0000 7ffa35ebd640 1 -- start start 2024-03-16T20:44:42.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa35ebd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa300b7540 0x7ffa30146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa35ebd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa300b7d40 0x7ffa3013a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa35ebd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa30148480 0x7ffa30136fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa35ebd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa3013c890 con 0x7ffa30148480 2024-03-16T20:44:42.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa35ebd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa301470b0 con 0x7ffa300b7d40 2024-03-16T20:44:42.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa35ebd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa30147a90 con 0x7ffa300b7540 2024-03-16T20:44:42.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa2effd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa300b7d40 0x7ffa3013a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa2f7fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa300b7540 0x7ffa30146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa2effd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa300b7d40 0x7ffa3013a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.195:39560/0 (socket says 172.21.15.195:39560) 2024-03-16T20:44:42.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa2effd640 1 -- 172.21.15.195:0/3941082472 learned_addr learned my addr 172.21.15.195:0/3941082472 (peer_addr_for_me v2:172.21.15.195:0/0) 2024-03-16T20:44:42.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa2ffff640 1 --2- 172.21.15.195:0/3941082472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa30148480 0x7ffa30136fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.597 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa2f7fe640 1 -- 172.21.15.195:0/3941082472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa300b7d40 msgr2=0x7ffa3013a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.598 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa2f7fe640 1 --2- 172.21.15.195:0/3941082472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa300b7d40 0x7ffa3013a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.598 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa2f7fe640 1 -- 172.21.15.195:0/3941082472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa30148480 msgr2=0x7ffa30136fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.598 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa2f7fe640 1 --2- 172.21.15.195:0/3941082472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa30148480 0x7ffa30136fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.598 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa2f7fe640 1 -- 172.21.15.195:0/3941082472 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa24009420 con 0x7ffa300b7540 2024-03-16T20:44:42.598 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa2effd640 1 --2- 172.21.15.195:0/3941082472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa300b7d40 0x7ffa3013a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:44:42.598 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa2f7fe640 1 --2- 172.21.15.195:0/3941082472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa300b7540 0x7ffa30146070 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000c430 tx=0x7ffa2000c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:42.598 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa17fff640 1 -- 172.21.15.195:0/3941082472 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa200074e0 con 0x7ffa300b7540 2024-03-16T20:44:42.598 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa17fff640 1 -- 172.21.15.195:0/3941082472 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa20007660 con 0x7ffa300b7540 2024-03-16T20:44:42.598 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa35ebd640 1 -- 172.21.15.195:0/3941082472 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa30147d70 con 0x7ffa300b7540 2024-03-16T20:44:42.598 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa17fff640 1 -- 172.21.15.195:0/3941082472 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa20004a30 con 0x7ffa300b7540 2024-03-16T20:44:42.598 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa35ebd640 1 -- 172.21.15.195:0/3941082472 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa30147320 con 0x7ffa300b7540 2024-03-16T20:44:42.598 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa17fff640 1 -- 172.21.15.195:0/3941082472 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ffa20004b90 con 0x7ffa300b7540 2024-03-16T20:44:42.598 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa17fff640 1 -- 172.21.15.195:0/3941082472 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7ffa20004e40 con 0x7ffa300b7540 2024-03-16T20:44:42.599 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.594+0000 7ffa35ebd640 1 -- 172.21.15.195:0/3941082472 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa301465b0 con 0x7ffa300b7540 2024-03-16T20:44:42.599 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.598+0000 7ffa17fff640 1 -- 172.21.15.195:0/3941082472 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ffa2001dbe0 con 0x7ffa300b7540 2024-03-16T20:44:42.663 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.662+0000 7ffa35ebd640 1 -- 172.21.15.195:0/3941082472 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd new", "uuid": "ac0af635-e34b-4876-a01d-d9c98833385e", "id": 7} v 0) v1 -- 0x7ffa300b2db0 con 0x7ffa300b7540 2024-03-16T20:44:42.670 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.666+0000 7ffa17fff640 1 -- 172.21.15.195:0/3941082472 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ac0af635-e34b-4876-a01d-d9c98833385e", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7ffa2001d4c0 con 0x7ffa300b7540 2024-03-16T20:44:42.670 INFO:teuthology.orchestra.run.smithi195.stdout:7 2024-03-16T20:44:42.671 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.670+0000 7ffa35ebd640 1 -- 172.21.15.195:0/3941082472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa300b7540 msgr2=0x7ffa30146070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.671 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.670+0000 7ffa35ebd640 1 --2- 172.21.15.195:0/3941082472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa300b7540 0x7ffa30146070 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffa2000c430 tx=0x7ffa2000c8f0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.671 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.670+0000 7ffa35ebd640 1 -- 172.21.15.195:0/3941082472 shutdown_connections 2024-03-16T20:44:42.671 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.670+0000 7ffa35ebd640 1 --2- 172.21.15.195:0/3941082472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffa30148480 0x7ffa30136fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.672 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.670+0000 7ffa35ebd640 1 --2- 172.21.15.195:0/3941082472 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ffa300b7d40 0x7ffa3013a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.672 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.670+0000 7ffa35ebd640 1 --2- 172.21.15.195:0/3941082472 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ffa300b7540 0x7ffa30146070 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.672 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.670+0000 7ffa35ebd640 1 -- 172.21.15.195:0/3941082472 >> 172.21.15.195:0/3941082472 conn(0x7ffa3006fc00 msgr2=0x7ffa300b4de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:42.672 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.670+0000 7ffa35ebd640 1 -- 172.21.15.195:0/3941082472 shutdown_connections 2024-03-16T20:44:42.672 INFO:teuthology.orchestra.run.smithi195.stderr:2024-03-16T20:44:42.670+0000 7ffa35ebd640 1 -- 172.21.15.195:0/3941082472 wait complete. 2024-03-16T20:44:42.686 INFO:tasks.ceph.osd.0:Restarting daemon 2024-03-16T20:44:42.686 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-03-16T20:44:42.689 INFO:tasks.ceph.osd.0:Started 2024-03-16T20:44:42.689 INFO:tasks.ceph.osd.1:Restarting daemon 2024-03-16T20:44:42.689 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-03-16T20:44:42.692 INFO:tasks.ceph.osd.1:Started 2024-03-16T20:44:42.692 INFO:tasks.ceph.osd.2:Restarting daemon 2024-03-16T20:44:42.692 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-03-16T20:44:42.694 INFO:tasks.ceph.osd.2:Started 2024-03-16T20:44:42.695 INFO:tasks.ceph.osd.3:Restarting daemon 2024-03-16T20:44:42.695 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-03-16T20:44:42.697 INFO:tasks.ceph.osd.3:Started 2024-03-16T20:44:42.697 INFO:tasks.ceph.osd.4:Restarting daemon 2024-03-16T20:44:42.698 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-03-16T20:44:42.700 INFO:tasks.ceph.osd.4:Started 2024-03-16T20:44:42.700 INFO:tasks.ceph.osd.5:Restarting daemon 2024-03-16T20:44:42.700 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-03-16T20:44:42.702 INFO:tasks.ceph.osd.5:Started 2024-03-16T20:44:42.702 INFO:tasks.ceph.osd.6:Restarting daemon 2024-03-16T20:44:42.702 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-03-16T20:44:42.704 INFO:tasks.ceph.osd.6:Started 2024-03-16T20:44:42.704 INFO:tasks.ceph.osd.7:Restarting daemon 2024-03-16T20:44:42.705 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-03-16T20:44:42.707 INFO:tasks.ceph.osd.7:Started 2024-03-16T20:44:42.707 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:44:42.726 INFO:tasks.ceph.mgr.x.smithi195.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-16T20:44:42.726 INFO:tasks.ceph.mgr.x.smithi195.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-16T20:44:42.726 INFO:tasks.ceph.mgr.x.smithi195.stderr: from numpy import show_config as show_numpy_config 2024-03-16T20:44:42.775 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:42.774+0000 7ff351d72180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T20:44:42.791 INFO:tasks.ceph.mgr.y.smithi122.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-03-16T20:44:42.791 INFO:tasks.ceph.mgr.y.smithi122.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-03-16T20:44:42.792 INFO:tasks.ceph.mgr.y.smithi122.stderr: from numpy import show_config as show_numpy_config 2024-03-16T20:44:42.834 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:42.831+0000 7fe73285d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-16T20:44:42.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.847+0000 7f556ef20640 1 Processor -- start 2024-03-16T20:44:42.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.847+0000 7f556ef20640 1 -- start start 2024-03-16T20:44:42.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.847+0000 7f556ef20640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f556813f3f0 0x7f5568150820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.847+0000 7f556ef20640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55680b1760 con 0x7f55680b64c0 2024-03-16T20:44:42.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.847+0000 7f556ef20640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55680b19b0 con 0x7f5568148f00 2024-03-16T20:44:42.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.847+0000 7f556ef20640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55680b3140 con 0x7f556813f3f0 2024-03-16T20:44:42.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.847+0000 7f556ccbc640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f55680b64c0 0x7f55681487f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:47940/0 (socket says 172.21.15.122:47940) 2024-03-16T20:44:42.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.847+0000 7f556ccbc640 1 -- 172.21.15.122:0/1086006809 learned_addr learned my addr 172.21.15.122:0/1086006809 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:44:42.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.847+0000 7f556d4bd640 1 --2- 172.21.15.122:0/1086006809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f556813f3f0 0x7f5568150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.847+0000 7f55677fe640 1 -- 172.21.15.122:0/1086006809 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 399314589 0 0) 0x7f55680b1760 con 0x7f55680b64c0 2024-03-16T20:44:42.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.847+0000 7f55677fe640 1 -- 172.21.15.122:0/1086006809 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5558003530 con 0x7f55680b64c0 2024-03-16T20:44:42.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f55677fe640 1 -- 172.21.15.122:0/1086006809 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3057185050 0 0) 0x7f5558003530 con 0x7f55680b64c0 2024-03-16T20:44:42.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f55677fe640 1 -- 172.21.15.122:0/1086006809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f556813f3f0 msgr2=0x7f5568150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:42.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f55677fe640 1 --2- 172.21.15.122:0/1086006809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f556813f3f0 0x7f5568150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f55677fe640 1 -- 172.21.15.122:0/1086006809 >> v1:172.21.15.195:6789/0 conn(0x7f5568148f00 legacy=0x7f556813ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f55677fe640 1 -- 172.21.15.122:0/1086006809 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55680b2ed0 con 0x7f55680b64c0 2024-03-16T20:44:42.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f55677fe640 1 -- 172.21.15.122:0/1086006809 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f555c001680 con 0x7f55680b64c0 2024-03-16T20:44:42.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f55677fe640 1 -- 172.21.15.122:0/1086006809 >> v1:172.21.15.122:6789/0 conn(0x7f55680b64c0 legacy=0x7f55681487f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f55677fe640 1 --2- 172.21.15.122:0/1086006809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55580039b0 0x7f5558003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f55677fe640 1 --2- 172.21.15.122:0/1086006809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f556813f3f0 0x7f55580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f55677fe640 1 --2- 172.21.15.122:0/1086006809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5558004d10 0x7f5558005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f55677fe640 1 -- 172.21.15.122:0/1086006809 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55680b3140 con 0x7f5558004d10 2024-03-16T20:44:42.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f55677fe640 1 -- 172.21.15.122:0/1086006809 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555800bb20 con 0x7f55580039b0 2024-03-16T20:44:42.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f55677fe640 1 -- 172.21.15.122:0/1086006809 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f555800bcb0 con 0x7f556813f3f0 2024-03-16T20:44:42.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f556d4bd640 1 --2- 172.21.15.122:0/1086006809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5558004d10 0x7f5558005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f556ccbc640 1 --2- 172.21.15.122:0/1086006809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55580039b0 0x7f5558003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f556d4bd640 1 -- 172.21.15.122:0/1086006809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f556813f3f0 msgr2=0x7f55580047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:42.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f556d4bd640 1 --2- 172.21.15.122:0/1086006809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f556813f3f0 0x7f55580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f556d4bd640 1 -- 172.21.15.122:0/1086006809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55580039b0 msgr2=0x7f5558003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:42.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f556d4bd640 1 --2- 172.21.15.122:0/1086006809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55580039b0 0x7f5558003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f556d4bd640 1 -- 172.21.15.122:0/1086006809 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5558003530 con 0x7f5558004d10 2024-03-16T20:44:42.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f556ccbc640 1 --2- 172.21.15.122:0/1086006809 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55580039b0 0x7f5558003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:44:42.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f556d4bd640 1 --2- 172.21.15.122:0/1086006809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5558004d10 0x7f5558005190 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f556000e170 tx=0x7f556001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30c8f170edddcae server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:42.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f55677fe640 1 -- 172.21.15.122:0/1086006809 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55600a5cb0 con 0x7f5558004d10 2024-03-16T20:44:42.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f55677fe640 1 -- 172.21.15.122:0/1086006809 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556000f040 con 0x7f5558004d10 2024-03-16T20:44:42.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f55677fe640 1 -- 172.21.15.122:0/1086006809 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55600ad680 con 0x7f5558004d10 2024-03-16T20:44:42.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f556ef20640 1 -- 172.21.15.122:0/1086006809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5558004d10 msgr2=0x7f5558005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f556ef20640 1 --2- 172.21.15.122:0/1086006809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5558004d10 0x7f5558005190 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f556000e170 tx=0x7f556001cca0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f556d4bd640 1 -- 172.21.15.122:0/1086006809 reap_dead start 2024-03-16T20:44:42.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f556ef20640 1 -- 172.21.15.122:0/1086006809 shutdown_connections 2024-03-16T20:44:42.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f556ef20640 1 -- 172.21.15.122:0/1086006809 >> 172.21.15.122:0/1086006809 conn(0x7f55680b7350 msgr2=0x7f55680b7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:42.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f556ef20640 1 -- 172.21.15.122:0/1086006809 shutdown_connections 2024-03-16T20:44:42.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.851+0000 7f556ef20640 1 -- 172.21.15.122:0/1086006809 wait complete. 2024-03-16T20:44:42.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ef20640 1 Processor -- start 2024-03-16T20:44:42.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ef20640 1 -- start start 2024-03-16T20:44:42.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ef20640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55680b9db0 0x7f5568143350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ef20640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55681dd730 0x7f55681345c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ef20640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5568134b00 0x7f5568134f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:42.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ef20640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55680b9010 con 0x7f55680b9db0 2024-03-16T20:44:42.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ef20640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55680b15c0 con 0x7f55681dd730 2024-03-16T20:44:42.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ef20640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55680ba5c0 con 0x7f5568134b00 2024-03-16T20:44:42.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ccbc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55680b9db0 0x7f5568143350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ccbc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55680b9db0 0x7f5568143350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49414/0 (socket says 172.21.15.122:49414) 2024-03-16T20:44:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ccbc640 1 -- 172.21.15.122:0/2667111131 learned_addr learned my addr 172.21.15.122:0/2667111131 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:44:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f5567fff640 1 --2- 172.21.15.122:0/2667111131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55681dd730 0x7f55681345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556d4bd640 1 --2- 172.21.15.122:0/2667111131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5568134b00 0x7f5568134f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ccbc640 1 -- 172.21.15.122:0/2667111131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5568134b00 msgr2=0x7f5568134f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ccbc640 1 --2- 172.21.15.122:0/2667111131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5568134b00 0x7f5568134f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ccbc640 1 -- 172.21.15.122:0/2667111131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55681dd730 msgr2=0x7f55681345c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ccbc640 1 --2- 172.21.15.122:0/2667111131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55681dd730 0x7f55681345c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ccbc640 1 -- 172.21.15.122:0/2667111131 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5568143b80 con 0x7f55680b9db0 2024-03-16T20:44:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ccbc640 1 --2- 172.21.15.122:0/2667111131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55680b9db0 0x7f5568143350 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f555c0017b0 tx=0x7f555c00df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f55657fa640 1 -- 172.21.15.122:0/2667111131 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c004cc0 con 0x7f55680b9db0 2024-03-16T20:44:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f55657fa640 1 -- 172.21.15.122:0/2667111131 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f555c004e20 con 0x7f55680b9db0 2024-03-16T20:44:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ef20640 1 -- 172.21.15.122:0/2667111131 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5568143e60 con 0x7f55680b9db0 2024-03-16T20:44:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f55657fa640 1 -- 172.21.15.122:0/2667111131 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555c0072d0 con 0x7f55680b9db0 2024-03-16T20:44:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ef20640 1 -- 172.21.15.122:0/2667111131 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55681ddbb0 con 0x7f55680b9db0 2024-03-16T20:44:42.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f556ef20640 1 -- 172.21.15.122:0/2667111131 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55680b1730 con 0x7f55680b9db0 2024-03-16T20:44:42.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f55657fa640 1 -- 172.21.15.122:0/2667111131 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f555c007430 con 0x7f55680b9db0 2024-03-16T20:44:42.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.855+0000 7f55657fa640 1 -- 172.21.15.122:0/2667111131 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f555c01eb10 con 0x7f55680b9db0 2024-03-16T20:44:42.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.863+0000 7f55657fa640 1 -- 172.21.15.122:0/2667111131 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f555c019070 con 0x7f55680b9db0 2024-03-16T20:44:42.907 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:42.906+0000 7ff351d72180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T20:44:42.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.919+0000 7f556ef20640 1 -- 172.21.15.122:0/2667111131 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5568075280 con 0x7f55680b9db0 2024-03-16T20:44:42.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.923+0000 7f55657fa640 1 -- 172.21.15.122:0/2667111131 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f555c01e440 con 0x7f55680b9db0 2024-03-16T20:44:42.925 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:42.925 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":10,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:44:42.665892+0000","last_up_change":"0.000000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:44:42.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.923+0000 7f556ef20640 1 -- 172.21.15.122:0/2667111131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55680b9db0 msgr2=0x7f5568143350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:42.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.923+0000 7f556ef20640 1 --2- 172.21.15.122:0/2667111131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55680b9db0 0x7f5568143350 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f555c0017b0 tx=0x7f555c00df70 comp rx=0 tx=0).stop 2024-03-16T20:44:42.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.923+0000 7f556ef20640 1 -- 172.21.15.122:0/2667111131 shutdown_connections 2024-03-16T20:44:42.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.923+0000 7f556ef20640 1 --2- 172.21.15.122:0/2667111131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5568134b00 0x7f5568134f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.923+0000 7f556ef20640 1 --2- 172.21.15.122:0/2667111131 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55681dd730 0x7f55681345c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.923+0000 7f556ef20640 1 --2- 172.21.15.122:0/2667111131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f55680b9db0 0x7f5568143350 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:42.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.923+0000 7f556ef20640 1 -- 172.21.15.122:0/2667111131 >> 172.21.15.122:0/2667111131 conn(0x7f55680b7350 msgr2=0x7f5568074320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:42.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.923+0000 7f556ef20640 1 -- 172.21.15.122:0/2667111131 shutdown_connections 2024-03-16T20:44:42.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:42.923+0000 7f556ef20640 1 -- 172.21.15.122:0/2667111131 wait complete. 2024-03-16T20:44:42.942 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-03-16T20:44:42.942 INFO:tasks.ceph:Waiting for OSDs to come up 2024-03-16T20:44:42.982 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:42.979+0000 7fe73285d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-16T20:44:43.048 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:43.046+0000 7ff351d72180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T20:44:43.065 INFO:tasks.ceph.osd.1.smithi122.stderr:2024-03-16T20:44:43.063+0000 7fc741f967c0 -1 Falling back to public interface 2024-03-16T20:44:43.113 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:43.110+0000 7ff351d72180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T20:44:43.162 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:43.159+0000 7fe73285d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-16T20:44:43.225 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:43.223+0000 7fe73285d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-16T20:44:43.245 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:43.242+0000 7ff351d72180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T20:44:43.309 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-03-16T20:44:43.307+0000 7f3619abc7c0 -1 Falling back to public interface 2024-03-16T20:44:43.311 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-03-16T20:44:43.310+0000 7f2668e5f7c0 -1 Falling back to public interface 2024-03-16T20:44:43.312 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-16T20:44:43.310+0000 7f86ab07e7c0 -1 Falling back to public interface 2024-03-16T20:44:43.313 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-03-16T20:44:43.310+0000 7f07af9e77c0 -1 Falling back to public interface 2024-03-16T20:44:43.327 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-16T20:44:43.326+0000 7f0987e9f7c0 -1 Falling back to public interface 2024-03-16T20:44:43.332 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-03-16T20:44:43.331+0000 7ff64061d7c0 -1 Falling back to public interface 2024-03-16T20:44:43.333 INFO:tasks.ceph.osd.2.smithi122.stderr:2024-03-16T20:44:43.331+0000 7fa990b0f7c0 -1 Falling back to public interface 2024-03-16T20:44:43.364 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:43.359+0000 7fe73285d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-16T20:44:43.747 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-16T20:44:43.856 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.851+0000 7f22924cc640 1 Processor -- start 2024-03-16T20:44:43.856 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.855+0000 7f22924cc640 1 -- start start 2024-03-16T20:44:43.857 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.855+0000 7f22924cc640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f228c141810 0x7f228c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:43.857 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.855+0000 7f22924cc640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228c0b5300 con 0x7f228c071470 2024-03-16T20:44:43.857 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.855+0000 7f22924cc640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228c06f1f0 con 0x7f228c0acf10 2024-03-16T20:44:43.857 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.855+0000 7f22924cc640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f228c0aca80 con 0x7f228c141810 2024-03-16T20:44:43.857 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.855+0000 7f2290a69640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f228c141810 0x7f228c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:43.857 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.855+0000 7f228bfff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f228c071470 0x7f228c071c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:47954/0 (socket says 172.21.15.122:47954) 2024-03-16T20:44:43.857 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.855+0000 7f228bfff640 1 -- 172.21.15.122:0/2135909474 learned_addr learned my addr 172.21.15.122:0/2135909474 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:44:43.857 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.855+0000 7f2290a69640 1 -- 172.21.15.122:0/2135909474 >> v1:172.21.15.195:6789/0 conn(0x7f228c0acf10 legacy=0x7f228c141100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:43.857 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.855+0000 7f2290a69640 1 -- 172.21.15.122:0/2135909474 >> v1:172.21.15.122:6789/0 conn(0x7f228c071470 legacy=0x7f228c071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:43.857 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.855+0000 7f2290a69640 1 -- 172.21.15.122:0/2135909474 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228c071850 con 0x7f228c141810 2024-03-16T20:44:43.859 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.855+0000 7f2290a69640 1 --2- 172.21.15.122:0/2135909474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c141810 0x7f228c0b4dc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2284009d90 tx=0x7f228401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa835566fead20de server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:43.859 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.855+0000 7f228affd640 1 -- 172.21.15.122:0/2135909474 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22840a5960 con 0x7f228c141810 2024-03-16T20:44:43.859 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.855+0000 7f228affd640 1 -- 172.21.15.122:0/2135909474 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22840a5ac0 con 0x7f228c141810 2024-03-16T20:44:43.859 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.855+0000 7f228affd640 1 -- 172.21.15.122:0/2135909474 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22840a5e50 con 0x7f228c141810 2024-03-16T20:44:43.860 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 -- 172.21.15.122:0/2135909474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c141810 msgr2=0x7f228c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:43.860 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 --2- 172.21.15.122:0/2135909474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c141810 0x7f228c0b4dc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2284009d90 tx=0x7f228401c8a0 comp rx=0 tx=0).stop 2024-03-16T20:44:43.860 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 -- 172.21.15.122:0/2135909474 shutdown_connections 2024-03-16T20:44:43.860 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 --2- 172.21.15.122:0/2135909474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c141810 0x7f228c0b4dc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:43.861 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 -- 172.21.15.122:0/2135909474 >> 172.21.15.122:0/2135909474 conn(0x7f228c06c0d0 msgr2=0x7f228c06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:43.861 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 -- 172.21.15.122:0/2135909474 shutdown_connections 2024-03-16T20:44:43.861 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 -- 172.21.15.122:0/2135909474 wait complete. 2024-03-16T20:44:43.863 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 Processor -- start 2024-03-16T20:44:43.863 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 -- start start 2024-03-16T20:44:43.863 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f228c071470 0x7f228c0afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:43.863 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f228c0acf10 0x7f228c1e31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:43.863 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c141810 0x7f228c139880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:43.863 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c143e60 con 0x7f228c141810 2024-03-16T20:44:43.863 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c0b0460 con 0x7f228c0acf10 2024-03-16T20:44:43.863 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228c0b1ea0 con 0x7f228c071470 2024-03-16T20:44:43.864 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f2290a69640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c141810 0x7f228c139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:43.864 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f2290a69640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c141810 0x7f228c139880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49432/0 (socket says 172.21.15.122:49432) 2024-03-16T20:44:43.864 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f2290a69640 1 -- 172.21.15.122:0/3752860854 learned_addr learned my addr 172.21.15.122:0/3752860854 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:44:43.864 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f228bfff640 1 --2- 172.21.15.122:0/3752860854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f228c071470 0x7f228c0afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:43.864 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f228b7fe640 1 --2- 172.21.15.122:0/3752860854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f228c0acf10 0x7f228c1e31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:43.864 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f2290a69640 1 -- 172.21.15.122:0/3752860854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f228c071470 msgr2=0x7f228c0afa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:43.864 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f2290a69640 1 --2- 172.21.15.122:0/3752860854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f228c071470 0x7f228c0afa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:43.864 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f2290a69640 1 -- 172.21.15.122:0/3752860854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f228c0acf10 msgr2=0x7f228c1e31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:43.864 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f2290a69640 1 --2- 172.21.15.122:0/3752860854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f228c0acf10 0x7f228c1e31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:43.864 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f2290a69640 1 -- 172.21.15.122:0/3752860854 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2284009a70 con 0x7f228c141810 2024-03-16T20:44:43.864 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f2290a69640 1 --2- 172.21.15.122:0/3752860854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c141810 0x7f228c139880 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f228401a4d0 tx=0x7f2284019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:43.864 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f2288ff9640 1 -- 172.21.15.122:0/3752860854 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22840addc0 con 0x7f228c141810 2024-03-16T20:44:43.865 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f2288ff9640 1 -- 172.21.15.122:0/3752860854 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22840b9450 con 0x7f228c141810 2024-03-16T20:44:43.865 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 -- 172.21.15.122:0/3752860854 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228c143660 con 0x7f228c141810 2024-03-16T20:44:43.865 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f2288ff9640 1 -- 172.21.15.122:0/3752860854 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22840b97e0 con 0x7f228c141810 2024-03-16T20:44:43.865 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 -- 172.21.15.122:0/3752860854 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228c0b1040 con 0x7f228c141810 2024-03-16T20:44:43.865 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f2288ff9640 1 -- 172.21.15.122:0/3752860854 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f22840b9cd0 con 0x7f228c141810 2024-03-16T20:44:43.865 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f2288ff9640 1 -- 172.21.15.122:0/3752860854 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f22840b7ad0 con 0x7f228c141810 2024-03-16T20:44:43.865 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.859+0000 7f22924cc640 1 -- 172.21.15.122:0/3752860854 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228c070f80 con 0x7f228c141810 2024-03-16T20:44:43.865 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:43.862+0000 7ff351d72180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T20:44:43.866 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.863+0000 7f2288ff9640 1 -- 172.21.15.122:0/3752860854 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f22840bd020 con 0x7f228c141810 2024-03-16T20:44:43.922 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.919+0000 7f22924cc640 1 -- 172.21.15.122:0/3752860854 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f228c075290 con 0x7f228c141810 2024-03-16T20:44:43.923 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.919+0000 7f2288ff9640 1 -- 172.21.15.122:0/3752860854 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f22840b2070 con 0x7f228c141810 2024-03-16T20:44:43.923 INFO:teuthology.misc.health.smithi122.stdout: 2024-03-16T20:44:43.923 INFO:teuthology.misc.health.smithi122.stdout:{"epoch":10,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:44:42.665892+0000","last_up_change":"0.000000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:44:43.924 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.919+0000 7f22924cc640 1 -- 172.21.15.122:0/3752860854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c141810 msgr2=0x7f228c139880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:43.924 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.919+0000 7f22924cc640 1 --2- 172.21.15.122:0/3752860854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c141810 0x7f228c139880 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f228401a4d0 tx=0x7f2284019640 comp rx=0 tx=0).stop 2024-03-16T20:44:43.924 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.919+0000 7f22924cc640 1 -- 172.21.15.122:0/3752860854 shutdown_connections 2024-03-16T20:44:43.924 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.919+0000 7f22924cc640 1 --2- 172.21.15.122:0/3752860854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f228c141810 0x7f228c139880 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:43.924 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.919+0000 7f22924cc640 1 --2- 172.21.15.122:0/3752860854 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f228c0acf10 0x7f228c1e31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:43.924 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.919+0000 7f22924cc640 1 --2- 172.21.15.122:0/3752860854 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f228c071470 0x7f228c0afa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:43.924 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.919+0000 7f22924cc640 1 -- 172.21.15.122:0/3752860854 >> 172.21.15.122:0/3752860854 conn(0x7f228c06c0d0 msgr2=0x7f228c140ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:43.925 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.919+0000 7f22924cc640 1 -- 172.21.15.122:0/3752860854 shutdown_connections 2024-03-16T20:44:43.925 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:43.923+0000 7f22924cc640 1 -- 172.21.15.122:0/3752860854 wait complete. 2024-03-16T20:44:43.938 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-16T20:44:43.941 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:43.938+0000 7ff351d72180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T20:44:43.974 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:43.971+0000 7fe73285d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-16T20:44:44.050 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:44.047+0000 7fe73285d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-16T20:44:44.122 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:44.118+0000 7ff351d72180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T20:44:44.195 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:44.194+0000 7ff351d72180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T20:44:44.229 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:44.227+0000 7fe73285d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-16T20:44:44.303 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:44.299+0000 7fe73285d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-16T20:44:44.483 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:44.482+0000 7ff351d72180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T20:44:44.545 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:44.542+0000 7ff351d72180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T20:44:44.593 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:44.590+0000 7fe73285d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-16T20:44:44.657 INFO:tasks.ceph.mgr.y.smithi122.stderr:2024-03-16T20:44:44.654+0000 7fe73285d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-16T20:44:45.593 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:45.590+0000 7ff2f91c1640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-03-16T20:44:45.593 INFO:tasks.ceph.mgr.x.smithi195.stderr:2024-03-16T20:44:45.590+0000 7ff2f91c1640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-16T20:44:46.927 INFO:tasks.ceph.osd.1.smithi122.stderr:2024-03-16T20:44:46.922+0000 7fc741f967c0 -1 osd.1 0 log_to_monitors true 2024-03-16T20:44:47.134 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-16T20:44:47.130+0000 7f86ab07e7c0 -1 osd.4 0 log_to_monitors true 2024-03-16T20:44:47.151 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-16T20:44:47.150+0000 7f0987e9f7c0 -1 osd.7 0 log_to_monitors true 2024-03-16T20:44:47.152 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-03-16T20:44:47.150+0000 7f2668e5f7c0 -1 osd.5 0 log_to_monitors true 2024-03-16T20:44:47.188 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-03-16T20:44:47.182+0000 7f3619abc7c0 -1 osd.0 0 log_to_monitors true 2024-03-16T20:44:47.190 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-03-16T20:44:47.186+0000 7ff64061d7c0 -1 osd.3 0 log_to_monitors true 2024-03-16T20:44:47.240 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-03-16T20:44:47.238+0000 7f07af9e77c0 -1 osd.6 0 log_to_monitors true 2024-03-16T20:44:47.276 INFO:tasks.ceph.osd.2.smithi122.stderr:2024-03-16T20:44:47.274+0000 7fa990b0f7c0 -1 osd.2 0 log_to_monitors true 2024-03-16T20:44:50.068 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-16T20:44:50.065+0000 7f0983625640 -1 osd.7 0 waiting for initial osdmap 2024-03-16T20:44:50.092 INFO:tasks.ceph.osd.7.smithi195.stderr:2024-03-16T20:44:50.089+0000 7f097bc6c640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T20:44:50.151 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-16T20:44:50.149+0000 7f86a6804640 -1 osd.4 0 waiting for initial osdmap 2024-03-16T20:44:50.176 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-03-16T20:44:50.173+0000 7f07ab16d640 -1 osd.6 0 waiting for initial osdmap 2024-03-16T20:44:50.179 INFO:tasks.ceph.osd.4.smithi195.stderr:2024-03-16T20:44:50.177+0000 7f869ee4b640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T20:44:50.194 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-03-16T20:44:50.189+0000 7f26645e5640 -1 osd.5 0 waiting for initial osdmap 2024-03-16T20:44:50.203 INFO:tasks.ceph.osd.6.smithi195.stderr:2024-03-16T20:44:50.201+0000 7f07a37b4640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T20:44:50.220 INFO:tasks.ceph.osd.5.smithi195.stderr:2024-03-16T20:44:50.217+0000 7f265cc2c640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T20:44:50.714 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-03-16T20:44:50.710+0000 7f3616a78640 -1 osd.0 0 waiting for initial osdmap 2024-03-16T20:44:50.744 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-16T20:44:50.747 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-03-16T20:44:50.742+0000 7f360d889640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T20:44:50.788 INFO:tasks.ceph.osd.2.smithi122.stderr:2024-03-16T20:44:50.786+0000 7fa98dacb640 -1 osd.2 0 waiting for initial osdmap 2024-03-16T20:44:50.825 INFO:tasks.ceph.osd.2.smithi122.stderr:2024-03-16T20:44:50.822+0000 7fa9848dc640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T20:44:50.876 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-03-16T20:44:50.870+0000 7ff63bda3640 -1 osd.3 0 waiting for initial osdmap 2024-03-16T20:44:50.886 INFO:tasks.ceph.osd.1.smithi122.stderr:2024-03-16T20:44:50.882+0000 7fc73d71c640 -1 osd.1 0 waiting for initial osdmap 2024-03-16T20:44:50.894 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.890+0000 7f104f0c2640 1 Processor -- start 2024-03-16T20:44:50.894 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.890+0000 7f104f0c2640 1 -- start start 2024-03-16T20:44:50.895 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.890+0000 7f104f0c2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f104809bc20 0x7f104813ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:50.895 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.890+0000 7f104f0c2640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f104813f360 0x7f104814f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:50.895 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.890+0000 7f104f0c2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1048072d30 con 0x7f1048067080 2024-03-16T20:44:50.895 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.890+0000 7f104f0c2640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f104813f7f0 con 0x7f104813f360 2024-03-16T20:44:50.895 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.890+0000 7f104f0c2640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10480b55e0 con 0x7f104809bc20 2024-03-16T20:44:50.895 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.890+0000 7f104d65f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f104813f360 0x7f104814f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:50.896 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.890+0000 7f104d65f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f104813f360 0x7f104814f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58712/0 (socket says 172.21.15.122:58712) 2024-03-16T20:44:50.896 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.890+0000 7f104d65f640 1 -- 172.21.15.122:0/1292080568 learned_addr learned my addr 172.21.15.122:0/1292080568 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:44:50.896 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.890+0000 7f104d65f640 1 -- 172.21.15.122:0/1292080568 >> v2:172.21.15.195:3300/0 conn(0x7f104809bc20 msgr2=0x7f104813ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:50.896 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.890+0000 7f104d65f640 1 --2- 172.21.15.122:0/1292080568 >> v2:172.21.15.195:3300/0 conn(0x7f104809bc20 0x7f104813ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:50.896 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.890+0000 7f104d65f640 1 -- 172.21.15.122:0/1292080568 >> v1:172.21.15.195:6789/0 conn(0x7f1048067080 legacy=0x7f1048144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:50.896 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.890+0000 7f104d65f640 1 -- 172.21.15.122:0/1292080568 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104809b240 con 0x7f104813f360 2024-03-16T20:44:50.898 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.894+0000 7f104d65f640 1 --2- 172.21.15.122:0/1292080568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f104813f360 0x7f104814f950 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f104806c5f0 tx=0x7f104001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1125c8c84aaaf6b3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:50.898 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.894+0000 7f10477fe640 1 -- 172.21.15.122:0/1292080568 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10400a5800 con 0x7f104813f360 2024-03-16T20:44:50.898 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.894+0000 7f10477fe640 1 -- 172.21.15.122:0/1292080568 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10400a5960 con 0x7f104813f360 2024-03-16T20:44:50.898 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.894+0000 7f10477fe640 1 -- 172.21.15.122:0/1292080568 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10400a5cf0 con 0x7f104813f360 2024-03-16T20:44:50.899 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.894+0000 7f104f0c2640 1 -- 172.21.15.122:0/1292080568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f104813f360 msgr2=0x7f104814f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:50.899 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.894+0000 7f104f0c2640 1 --2- 172.21.15.122:0/1292080568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f104813f360 0x7f104814f950 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f104806c5f0 tx=0x7f104001c920 comp rx=0 tx=0).stop 2024-03-16T20:44:50.899 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.894+0000 7f104f0c2640 1 -- 172.21.15.122:0/1292080568 shutdown_connections 2024-03-16T20:44:50.899 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.894+0000 7f104f0c2640 1 --2- 172.21.15.122:0/1292080568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f104813f360 0x7f104814f950 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:50.899 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.894+0000 7f104f0c2640 1 --2- 172.21.15.122:0/1292080568 >> v2:172.21.15.195:3300/0 conn(0x7f104809bc20 0x7f104813ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:50.899 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.894+0000 7f104f0c2640 1 -- 172.21.15.122:0/1292080568 >> 172.21.15.122:0/1292080568 conn(0x7f104806d2c0 msgr2=0x7f104806d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:50.899 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.894+0000 7f104f0c2640 1 -- 172.21.15.122:0/1292080568 shutdown_connections 2024-03-16T20:44:50.899 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.894+0000 7f104f0c2640 1 -- 172.21.15.122:0/1292080568 wait complete. 2024-03-16T20:44:50.900 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.894+0000 7f104f0c2640 1 Processor -- start 2024-03-16T20:44:50.900 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.894+0000 7f104f0c2640 1 -- start start 2024-03-16T20:44:50.902 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f104f0c2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1048067080 0x7f10480ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:50.902 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f104f0c2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f104813f360 0x7f10481e3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:50.902 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f104f0c2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10481e3580 0x7f104809dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:50.902 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f104f0c2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10480af690 con 0x7f104813f360 2024-03-16T20:44:50.902 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f104f0c2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10481e3a10 con 0x7f1048067080 2024-03-16T20:44:50.902 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f104f0c2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10480aff50 con 0x7f10481e3580 2024-03-16T20:44:50.902 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f1047fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f104813f360 0x7f10481e3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:50.902 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f1047fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f104813f360 0x7f10481e3040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58716/0 (socket says 172.21.15.122:58716) 2024-03-16T20:44:50.902 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f1047fff640 1 -- 172.21.15.122:0/1184340909 learned_addr learned my addr 172.21.15.122:0/1184340909 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:44:50.902 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f104d65f640 1 --2- 172.21.15.122:0/1184340909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10481e3580 0x7f104809dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:50.902 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f1047fff640 1 -- 172.21.15.122:0/1184340909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10481e3580 msgr2=0x7f104809dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:50.902 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f104ce5e640 1 --2- 172.21.15.122:0/1184340909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1048067080 0x7f10480ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:50.903 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f1047fff640 1 --2- 172.21.15.122:0/1184340909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10481e3580 0x7f104809dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:50.903 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f1047fff640 1 -- 172.21.15.122:0/1184340909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1048067080 msgr2=0x7f10480ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:50.903 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f1047fff640 1 --2- 172.21.15.122:0/1184340909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1048067080 0x7f10480ae2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:50.903 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f1047fff640 1 -- 172.21.15.122:0/1184340909 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1040009b30 con 0x7f104813f360 2024-03-16T20:44:50.903 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f1047fff640 1 --2- 172.21.15.122:0/1184340909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f104813f360 0x7f10481e3040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f103400b4b0 tx=0x7f103400b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:50.903 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f10457fa640 1 -- 172.21.15.122:0/1184340909 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034010610 con 0x7f104813f360 2024-03-16T20:44:50.903 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f10457fa640 1 -- 172.21.15.122:0/1184340909 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1034010770 con 0x7f104813f360 2024-03-16T20:44:50.903 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f10457fa640 1 -- 172.21.15.122:0/1184340909 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034010b20 con 0x7f104813f360 2024-03-16T20:44:50.903 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f104f0c2640 1 -- 172.21.15.122:0/1184340909 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1048142a00 con 0x7f104813f360 2024-03-16T20:44:50.903 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f104f0c2640 1 -- 172.21.15.122:0/1184340909 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104809d330 con 0x7f104813f360 2024-03-16T20:44:50.904 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f104f0c2640 1 -- 172.21.15.122:0/1184340909 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10480afd90 con 0x7f104813f360 2024-03-16T20:44:50.904 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f10457fa640 1 -- 172.21.15.122:0/1184340909 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 4) v1 ==== 86353+0+0 (secure 0 0 0) 0x7f1034010c80 con 0x7f104813f360 2024-03-16T20:44:50.904 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f10457fa640 1 --2- 172.21.15.122:0/1184340909 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f102c066db0 0x7f102c069270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:50.904 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f10457fa640 1 -- 172.21.15.122:0/1184340909 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f10340836e0 con 0x7f104813f360 2024-03-16T20:44:50.904 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.898+0000 7f104ce5e640 1 --2- 172.21.15.122:0/1184340909 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f102c066db0 0x7f102c069270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:50.904 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.902+0000 7f104ce5e640 1 --2- 172.21.15.122:0/1184340909 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f102c066db0 0x7f102c069270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f103c001600 tx=0x7f103c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:50.908 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:50.902+0000 7f10457fa640 1 -- 172.21.15.122:0/1184340909 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f10340513d0 con 0x7f104813f360 2024-03-16T20:44:50.913 INFO:tasks.ceph.osd.3.smithi122.stderr:2024-03-16T20:44:50.910+0000 7ff6343ea640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T20:44:50.917 INFO:tasks.ceph.osd.1.smithi122.stderr:2024-03-16T20:44:50.914+0000 7fc735d63640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-16T20:44:51.066 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:51.062+0000 7f104f0c2640 1 -- 172.21.15.122:0/1184340909 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f104809a670 con 0x7f104813f360 2024-03-16T20:44:51.067 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:51.062+0000 7f10457fa640 1 -- 172.21.15.122:0/1184340909 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f1034014090 con 0x7f104813f360 2024-03-16T20:44:51.067 INFO:teuthology.misc.health.smithi122.stdout: 2024-03-16T20:44:51.067 INFO:teuthology.misc.health.smithi122.stdout:{"epoch":12,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:44:48.589772+0000","last_up_change":"0.000000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:44:51.069 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:51.066+0000 7f104f0c2640 1 -- 172.21.15.122:0/1184340909 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f102c066db0 msgr2=0x7f102c069270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:51.069 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:51.066+0000 7f104f0c2640 1 --2- 172.21.15.122:0/1184340909 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f102c066db0 0x7f102c069270 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f103c001600 tx=0x7f103c009040 comp rx=0 tx=0).stop 2024-03-16T20:44:51.069 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:51.066+0000 7f104f0c2640 1 -- 172.21.15.122:0/1184340909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f104813f360 msgr2=0x7f10481e3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:51.070 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:51.066+0000 7f104f0c2640 1 --2- 172.21.15.122:0/1184340909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f104813f360 0x7f10481e3040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f103400b4b0 tx=0x7f103400b970 comp rx=0 tx=0).stop 2024-03-16T20:44:51.070 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:51.066+0000 7f104f0c2640 1 -- 172.21.15.122:0/1184340909 shutdown_connections 2024-03-16T20:44:51.070 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:51.066+0000 7f104f0c2640 1 --2- 172.21.15.122:0/1184340909 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f102c066db0 0x7f102c069270 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:51.070 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:51.066+0000 7f104f0c2640 1 --2- 172.21.15.122:0/1184340909 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f10481e3580 0x7f104809dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:51.070 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:51.066+0000 7f104f0c2640 1 --2- 172.21.15.122:0/1184340909 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f104813f360 0x7f10481e3040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:51.070 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:51.066+0000 7f104f0c2640 1 --2- 172.21.15.122:0/1184340909 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1048067080 0x7f10480ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:51.070 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:51.066+0000 7f104f0c2640 1 -- 172.21.15.122:0/1184340909 >> 172.21.15.122:0/1184340909 conn(0x7f104806d2c0 msgr2=0x7f1048071b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:51.070 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:51.066+0000 7f104f0c2640 1 -- 172.21.15.122:0/1184340909 shutdown_connections 2024-03-16T20:44:51.070 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:51.066+0000 7f104f0c2640 1 -- 172.21.15.122:0/1184340909 wait complete. 2024-03-16T20:44:51.089 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-03-16T20:44:57.896 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-03-16T20:44:58.016 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.009+0000 7f8ac2950640 1 Processor -- start 2024-03-16T20:44:58.016 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.009+0000 7f8ac2950640 1 -- start start 2024-03-16T20:44:58.017 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8ac2950640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8abc14dac0 0x7f8abc14feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.018 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8ac2950640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc0b2b70 con 0x7f8abc145180 2024-03-16T20:44:58.018 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8ac2950640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc072480 con 0x7f8abc0720a0 2024-03-16T20:44:58.018 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8ac2950640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8abc0b1ab0 con 0x7f8abc14dac0 2024-03-16T20:44:58.018 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8ab3fff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f8abc145180 0x7f8abc145610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:54726/0 (socket says 172.21.15.122:54726) 2024-03-16T20:44:58.018 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8ab3fff640 1 -- 172.21.15.122:0/59631856 learned_addr learned my addr 172.21.15.122:0/59631856 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:44:58.018 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8ac0eed640 1 --2- 172.21.15.122:0/59631856 >> v2:172.21.15.195:3300/0 conn(0x7f8abc14dac0 0x7f8abc14feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.018 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8abb7fe640 1 -- 172.21.15.122:0/59631856 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898712936 0 0) 0x7f8abc0b2b70 con 0x7f8abc145180 2024-03-16T20:44:58.018 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8abb7fe640 1 -- 172.21.15.122:0/59631856 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aa8003530 con 0x7f8abc145180 2024-03-16T20:44:58.019 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8abb7fe640 1 -- 172.21.15.122:0/59631856 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2862085060 0 0) 0x7f8aa8003530 con 0x7f8abc145180 2024-03-16T20:44:58.019 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8abb7fe640 1 -- 172.21.15.122:0/59631856 >> v2:172.21.15.195:3300/0 conn(0x7f8abc14dac0 msgr2=0x7f8abc14feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:58.020 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8abb7fe640 1 --2- 172.21.15.122:0/59631856 >> v2:172.21.15.195:3300/0 conn(0x7f8abc14dac0 0x7f8abc14feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.020 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8abb7fe640 1 -- 172.21.15.122:0/59631856 >> v1:172.21.15.195:6789/0 conn(0x7f8abc0720a0 legacy=0x7f8abc0afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.020 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8abb7fe640 1 -- 172.21.15.122:0/59631856 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc0b1d90 con 0x7f8abc145180 2024-03-16T20:44:58.020 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8abb7fe640 1 -- 172.21.15.122:0/59631856 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f8aa4001680 con 0x7f8abc145180 2024-03-16T20:44:58.020 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8abb7fe640 1 -- 172.21.15.122:0/59631856 >> v1:172.21.15.122:6789/0 conn(0x7f8abc145180 legacy=0x7f8abc145610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.020 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8abb7fe640 1 --2- 172.21.15.122:0/59631856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8aa8003980 0x7f8aa8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.020 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8abb7fe640 1 --2- 172.21.15.122:0/59631856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8aa80047d0 0x7f8aa8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.020 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8abb7fe640 1 --2- 172.21.15.122:0/59631856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa8005190 0x7f8aa800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.020 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8abb7fe640 1 -- 172.21.15.122:0/59631856 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0b1ab0 con 0x7f8aa8005190 2024-03-16T20:44:58.020 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8abb7fe640 1 -- 172.21.15.122:0/59631856 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa800c130 con 0x7f8aa80047d0 2024-03-16T20:44:58.020 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8abb7fe640 1 -- 172.21.15.122:0/59631856 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa800c2f0 con 0x7f8aa8003980 2024-03-16T20:44:58.020 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8ac0eed640 1 --2- 172.21.15.122:0/59631856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa8005190 0x7f8aa800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.020 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8abbfff640 1 --2- 172.21.15.122:0/59631856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8aa8003980 0x7f8aa8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.020 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8ab3fff640 1 --2- 172.21.15.122:0/59631856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8aa80047d0 0x7f8aa8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.020 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8ac0eed640 1 -- 172.21.15.122:0/59631856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8aa8003980 msgr2=0x7f8aa8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:58.021 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8ac0eed640 1 --2- 172.21.15.122:0/59631856 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8aa8003980 0x7f8aa8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.021 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8ac0eed640 1 -- 172.21.15.122:0/59631856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8aa80047d0 msgr2=0x7f8aa8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:58.021 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8ac0eed640 1 --2- 172.21.15.122:0/59631856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8aa80047d0 0x7f8aa8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.021 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8ac0eed640 1 -- 172.21.15.122:0/59631856 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa8003530 con 0x7f8aa8005190 2024-03-16T20:44:58.021 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.013+0000 7f8ab3fff640 1 --2- 172.21.15.122:0/59631856 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8aa80047d0 0x7f8aa8004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:44:58.022 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac0eed640 1 -- 172.21.15.122:0/59631856 reap_dead start 2024-03-16T20:44:58.022 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac0eed640 1 --2- 172.21.15.122:0/59631856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa8005190 0x7f8aa800b990 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009fc0 tx=0x7f8ab401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63dfc3a55fcdab62 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:58.022 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8abb7fe640 1 -- 172.21.15.122:0/59631856 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40a5cd0 con 0x7f8aa8005190 2024-03-16T20:44:58.022 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8abb7fe640 1 -- 172.21.15.122:0/59631856 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa4001d80 con 0x7f8aa8005190 2024-03-16T20:44:58.022 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8abb7fe640 1 -- 172.21.15.122:0/59631856 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4003be0 con 0x7f8aa8005190 2024-03-16T20:44:58.022 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac2950640 1 -- 172.21.15.122:0/59631856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa8005190 msgr2=0x7f8aa800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.022 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac2950640 1 --2- 172.21.15.122:0/59631856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa8005190 0x7f8aa800b990 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009fc0 tx=0x7f8ab401ccc0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.022 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac2950640 1 -- 172.21.15.122:0/59631856 shutdown_connections 2024-03-16T20:44:58.023 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac2950640 1 --2- 172.21.15.122:0/59631856 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa8005190 0x7f8aa800b990 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.023 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac2950640 1 -- 172.21.15.122:0/59631856 >> 172.21.15.122:0/59631856 conn(0x7f8abc0b8850 msgr2=0x7f8abc0b8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:58.023 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac2950640 1 -- 172.21.15.122:0/59631856 shutdown_connections 2024-03-16T20:44:58.023 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac2950640 1 -- 172.21.15.122:0/59631856 wait complete. 2024-03-16T20:44:58.025 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac2950640 1 Processor -- start 2024-03-16T20:44:58.025 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac2950640 1 -- start start 2024-03-16T20:44:58.025 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac2950640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa8005190 0x7f8abc13a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.025 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac2950640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8abc13ae90 0x7f8abc1dda90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.025 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac2950640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8abc1ddfd0 0x7f8abc1da260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.025 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac2950640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc1413d0 con 0x7f8aa8005190 2024-03-16T20:44:58.025 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac2950640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc141620 con 0x7f8abc13ae90 2024-03-16T20:44:58.025 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac2950640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc13b310 con 0x7f8abc1ddfd0 2024-03-16T20:44:58.025 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa8005190 0x7f8abc13a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.025 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8abbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa8005190 0x7f8abc13a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:58742/0 (socket says 172.21.15.122:58742) 2024-03-16T20:44:58.026 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8abbfff640 1 -- 172.21.15.122:0/4220921902 learned_addr learned my addr 172.21.15.122:0/4220921902 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:44:58.026 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ab3fff640 1 --2- 172.21.15.122:0/4220921902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8abc13ae90 0x7f8abc1dda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.026 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac0eed640 1 --2- 172.21.15.122:0/4220921902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8abc1ddfd0 0x7f8abc1da260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.026 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8abbfff640 1 -- 172.21.15.122:0/4220921902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8abc1ddfd0 msgr2=0x7f8abc1da260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.026 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8abbfff640 1 --2- 172.21.15.122:0/4220921902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8abc1ddfd0 0x7f8abc1da260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.026 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8abbfff640 1 -- 172.21.15.122:0/4220921902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8abc13ae90 msgr2=0x7f8abc1dda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.026 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8abbfff640 1 --2- 172.21.15.122:0/4220921902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8abc13ae90 0x7f8abc1dda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.026 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8abbfff640 1 -- 172.21.15.122:0/4220921902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc141ef0 con 0x7f8aa8005190 2024-03-16T20:44:58.026 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ab3fff640 1 --2- 172.21.15.122:0/4220921902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8abc13ae90 0x7f8abc1dda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:44:58.026 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8abbfff640 1 --2- 172.21.15.122:0/4220921902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa8005190 0x7f8abc13a950 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00e440 tx=0x7f8aac00e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:58.026 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ab97fa640 1 -- 172.21.15.122:0/4220921902 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac0052e0 con 0x7f8aa8005190 2024-03-16T20:44:58.026 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ab97fa640 1 -- 172.21.15.122:0/4220921902 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aac005440 con 0x7f8aa8005190 2024-03-16T20:44:58.027 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac2950640 1 -- 172.21.15.122:0/4220921902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc1410b0 con 0x7f8aa8005190 2024-03-16T20:44:58.027 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ab97fa640 1 -- 172.21.15.122:0/4220921902 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac003820 con 0x7f8aa8005190 2024-03-16T20:44:58.027 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.017+0000 7f8ac2950640 1 -- 172.21.15.122:0/4220921902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc1423f0 con 0x7f8aa8005190 2024-03-16T20:44:58.027 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.021+0000 7f8ac2950640 1 -- 172.21.15.122:0/4220921902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8abc075290 con 0x7f8aa8005190 2024-03-16T20:44:58.028 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.021+0000 7f8ab97fa640 1 -- 172.21.15.122:0/4220921902 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f8aac003980 con 0x7f8aa8005190 2024-03-16T20:44:58.028 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.021+0000 7f8ab97fa640 1 --2- 172.21.15.122:0/4220921902 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8a9c066e40 0x7f8a9c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.028 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.021+0000 7f8ab97fa640 1 -- 172.21.15.122:0/4220921902 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8aac084360 con 0x7f8aa8005190 2024-03-16T20:44:58.028 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.021+0000 7f8ab3fff640 1 --2- 172.21.15.122:0/4220921902 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8a9c066e40 0x7f8a9c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.028 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.021+0000 7f8ab3fff640 1 --2- 172.21.15.122:0/4220921902 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8a9c066e40 0x7f8a9c069300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4000fc0 tx=0x7f8aa4001500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:58.031 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.025+0000 7f8ab97fa640 1 -- 172.21.15.122:0/4220921902 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8aac051530 con 0x7f8aa8005190 2024-03-16T20:44:58.190 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.185+0000 7f8ac2950640 1 -- 172.21.15.122:0/4220921902 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8abc142080 con 0x7f8aa8005190 2024-03-16T20:44:58.191 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.185+0000 7f8ab97fa640 1 -- 172.21.15.122:0/4220921902 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12920 (secure 0 0 0) 0x7f8abc142080 con 0x7f8aa8005190 2024-03-16T20:44:58.191 INFO:teuthology.misc.health.smithi122.stdout: 2024-03-16T20:44:58.191 INFO:teuthology.misc.health.smithi122.stdout:{"epoch":17,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:44:55.098139+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:44:58.194 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.189+0000 7f8ac2950640 1 -- 172.21.15.122:0/4220921902 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8a9c066e40 msgr2=0x7f8a9c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.194 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.189+0000 7f8ac2950640 1 --2- 172.21.15.122:0/4220921902 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8a9c066e40 0x7f8a9c069300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4000fc0 tx=0x7f8aa4001500 comp rx=0 tx=0).stop 2024-03-16T20:44:58.194 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.189+0000 7f8ac2950640 1 -- 172.21.15.122:0/4220921902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa8005190 msgr2=0x7f8abc13a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.194 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.189+0000 7f8ac2950640 1 --2- 172.21.15.122:0/4220921902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa8005190 0x7f8abc13a950 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00e440 tx=0x7f8aac00e900 comp rx=0 tx=0).stop 2024-03-16T20:44:58.194 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.189+0000 7f8ac2950640 1 -- 172.21.15.122:0/4220921902 shutdown_connections 2024-03-16T20:44:58.194 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.189+0000 7f8ac2950640 1 --2- 172.21.15.122:0/4220921902 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8a9c066e40 0x7f8a9c069300 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.194 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.189+0000 7f8ac2950640 1 --2- 172.21.15.122:0/4220921902 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8abc1ddfd0 0x7f8abc1da260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.194 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.189+0000 7f8ac2950640 1 --2- 172.21.15.122:0/4220921902 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8abc13ae90 0x7f8abc1dda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.195 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.189+0000 7f8ac2950640 1 --2- 172.21.15.122:0/4220921902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8aa8005190 0x7f8abc13a950 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.195 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.189+0000 7f8ac2950640 1 -- 172.21.15.122:0/4220921902 >> 172.21.15.122:0/4220921902 conn(0x7f8abc0b8850 msgr2=0x7f8abc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:58.195 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.189+0000 7f8ac2950640 1 -- 172.21.15.122:0/4220921902 shutdown_connections 2024-03-16T20:44:58.195 INFO:teuthology.misc.health.smithi122.stderr:2024-03-16T20:44:58.189+0000 7f8ac2950640 1 -- 172.21.15.122:0/4220921902 wait complete. 2024-03-16T20:44:58.210 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-03-16T20:44:58.210 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-03-16T20:44:58.210 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-16T20:44:58.210 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-16T20:44:58.212 INFO:tasks.ceph.mds.a:Started 2024-03-16T20:44:58.212 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-16T20:44:58.212 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-16T20:44:58.215 INFO:tasks.ceph.mds.b:Started 2024-03-16T20:44:58.215 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-03-16T20:44:58.216 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:44:58.287 INFO:tasks.ceph.mds.a.smithi122.stdout:starting mds.a at 2024-03-16T20:44:58.309 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-03-16T20:44:58.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.321+0000 7f30be729640 1 Processor -- start 2024-03-16T20:44:58.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.321+0000 7f30be729640 1 -- start start 2024-03-16T20:44:58.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30be729640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f30b8071a10 0x7f30b81484c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30be729640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30b8071df0 0x7f30b8148a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30be729640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b80b39f0 con 0x7f30b813ecd0 2024-03-16T20:44:58.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30be729640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b80b3680 con 0x7f30b8071a10 2024-03-16T20:44:58.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30be729640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80b3390 con 0x7f30b8071df0 2024-03-16T20:44:58.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30bccc6640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f30b813ecd0 0x7f30b813f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:33992/0 (socket says 172.21.15.122:33992) 2024-03-16T20:44:58.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30bccc6640 1 -- 172.21.15.122:0/1630084633 learned_addr learned my addr 172.21.15.122:0/1630084633 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:44:58.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30b77fe640 1 --2- 172.21.15.122:0/1630084633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30b8071df0 0x7f30b8148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30b7fff640 1 --2- 172.21.15.122:0/1630084633 >> v2:172.21.15.195:3300/0 conn(0x7f30b8071a10 0x7f30b81484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30b6ffd640 1 -- 172.21.15.122:0/1630084633 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1116404210 0 0) 0x7f30b80b39f0 con 0x7f30b813ecd0 2024-03-16T20:44:58.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30b6ffd640 1 -- 172.21.15.122:0/1630084633 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30a0003530 con 0x7f30b813ecd0 2024-03-16T20:44:58.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30b77fe640 1 -- 172.21.15.122:0/1630084633 >> v2:172.21.15.195:3300/0 conn(0x7f30b8071a10 msgr2=0x7f30b81484c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:58.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30b77fe640 1 --2- 172.21.15.122:0/1630084633 >> v2:172.21.15.195:3300/0 conn(0x7f30b8071a10 0x7f30b81484c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30b77fe640 1 -- 172.21.15.122:0/1630084633 >> v1:172.21.15.122:6789/0 conn(0x7f30b813ecd0 legacy=0x7f30b813f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30b77fe640 1 -- 172.21.15.122:0/1630084633 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b80b6d90 con 0x7f30b8071df0 2024-03-16T20:44:58.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30b77fe640 1 --2- 172.21.15.122:0/1630084633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30b8071df0 0x7f30b8148a00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0097c0 tx=0x7f30ac01c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e670d8245b3ca4ea server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:58.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30b6ffd640 1 -- 172.21.15.122:0/1630084633 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0aa620 con 0x7f30b8071df0 2024-03-16T20:44:58.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30b6ffd640 1 -- 172.21.15.122:0/1630084633 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac0aa780 con 0x7f30b8071df0 2024-03-16T20:44:58.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30b6ffd640 1 -- 172.21.15.122:0/1630084633 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0aab10 con 0x7f30b8071df0 2024-03-16T20:44:58.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30be729640 1 -- 172.21.15.122:0/1630084633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30b8071df0 msgr2=0x7f30b8148a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30be729640 1 --2- 172.21.15.122:0/1630084633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30b8071df0 0x7f30b8148a00 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f30ac0097c0 tx=0x7f30ac01c4a0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30be729640 1 -- 172.21.15.122:0/1630084633 shutdown_connections 2024-03-16T20:44:58.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30be729640 1 --2- 172.21.15.122:0/1630084633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30b8071df0 0x7f30b8148a00 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30be729640 1 --2- 172.21.15.122:0/1630084633 >> v2:172.21.15.195:3300/0 conn(0x7f30b8071a10 0x7f30b81484c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30be729640 1 -- 172.21.15.122:0/1630084633 >> 172.21.15.122:0/1630084633 conn(0x7f30b80b62a0 msgr2=0x7f30b80b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:58.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30be729640 1 -- 172.21.15.122:0/1630084633 shutdown_connections 2024-03-16T20:44:58.334 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.325+0000 7f30be729640 1 -- 172.21.15.122:0/1630084633 wait complete. 2024-03-16T20:44:58.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30be729640 1 Processor -- start 2024-03-16T20:44:58.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30be729640 1 -- start start 2024-03-16T20:44:58.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30be729640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30b8071a10 0x7f30b81350f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30be729640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b8071df0 0x7f30b81460b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30be729640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30b813ecd0 0x7f30b812f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30be729640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b814ee00 con 0x7f30b8071df0 2024-03-16T20:44:58.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30be729640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80b4a00 con 0x7f30b8071a10 2024-03-16T20:44:58.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30be729640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b8135b80 con 0x7f30b813ecd0 2024-03-16T20:44:58.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30b77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b8071df0 0x7f30b81460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30b77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b8071df0 0x7f30b81460b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33198/0 (socket says 172.21.15.122:33198) 2024-03-16T20:44:58.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30b77fe640 1 -- 172.21.15.122:0/3889093154 learned_addr learned my addr 172.21.15.122:0/3889093154 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:44:58.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30bccc6640 1 --2- 172.21.15.122:0/3889093154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30b813ecd0 0x7f30b812f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30b7fff640 1 --2- 172.21.15.122:0/3889093154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30b8071a10 0x7f30b81350f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30b77fe640 1 -- 172.21.15.122:0/3889093154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30b813ecd0 msgr2=0x7f30b812f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30b77fe640 1 --2- 172.21.15.122:0/3889093154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30b813ecd0 0x7f30b812f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30b77fe640 1 -- 172.21.15.122:0/3889093154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30b8071a10 msgr2=0x7f30b81350f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30b77fe640 1 --2- 172.21.15.122:0/3889093154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30b8071a10 0x7f30b81350f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30b77fe640 1 -- 172.21.15.122:0/3889093154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac009420 con 0x7f30b8071df0 2024-03-16T20:44:58.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30b7fff640 1 --2- 172.21.15.122:0/3889093154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30b8071a10 0x7f30b81350f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:44:58.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30b77fe640 1 --2- 172.21.15.122:0/3889093154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b8071df0 0x7f30b81460b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f30ac01cd40 tx=0x7f30ac01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:58.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30b4ff9640 1 -- 172.21.15.122:0/3889093154 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0aad00 con 0x7f30b8071df0 2024-03-16T20:44:58.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30b4ff9640 1 -- 172.21.15.122:0/3889093154 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac0aae60 con 0x7f30b8071df0 2024-03-16T20:44:58.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30b4ff9640 1 -- 172.21.15.122:0/3889093154 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0b4620 con 0x7f30b8071df0 2024-03-16T20:44:58.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30be729640 1 -- 172.21.15.122:0/3889093154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b814e8e0 con 0x7f30b8071df0 2024-03-16T20:44:58.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.329+0000 7f30be729640 1 -- 172.21.15.122:0/3889093154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b80b86d0 con 0x7f30b8071df0 2024-03-16T20:44:58.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.333+0000 7f30be729640 1 -- 172.21.15.122:0/3889093154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b8135970 con 0x7f30b8071df0 2024-03-16T20:44:58.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.333+0000 7f30b4ff9640 1 -- 172.21.15.122:0/3889093154 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f30ac0b4780 con 0x7f30b8071df0 2024-03-16T20:44:58.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.333+0000 7f30b4ff9640 1 --2- 172.21.15.122:0/3889093154 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f30a4066e40 0x7f30a4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.333+0000 7f30b4ff9640 1 -- 172.21.15.122:0/3889093154 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f30ac11e790 con 0x7f30b8071df0 2024-03-16T20:44:58.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.333+0000 7f30b7fff640 1 --2- 172.21.15.122:0/3889093154 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f30a4066e40 0x7f30a4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.333+0000 7f30b7fff640 1 --2- 172.21.15.122:0/3889093154 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f30a4066e40 0x7f30a4069300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f30b0009b10 tx=0x7f30b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:58.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.337+0000 7f30b4ff9640 1 -- 172.21.15.122:0/3889093154 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f30ac0ebaa0 con 0x7f30b8071df0 2024-03-16T20:44:58.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.489+0000 7f30be729640 1 -- 172.21.15.122:0/3889093154 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30b80b2fe0 con 0x7f30b8071df0 2024-03-16T20:44:58.496 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.489+0000 7f30b4ff9640 1 -- 172.21.15.122:0/3889093154 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12920 (secure 0 0 0) 0x7f30ac0c3020 con 0x7f30b8071df0 2024-03-16T20:44:58.496 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:58.496 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":17,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:44:55.098139+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:44:58.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.493+0000 7f30be729640 1 -- 172.21.15.122:0/3889093154 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f30a4066e40 msgr2=0x7f30a4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.493+0000 7f30be729640 1 --2- 172.21.15.122:0/3889093154 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f30a4066e40 0x7f30a4069300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f30b0009b10 tx=0x7f30b0008040 comp rx=0 tx=0).stop 2024-03-16T20:44:58.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.493+0000 7f30be729640 1 -- 172.21.15.122:0/3889093154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b8071df0 msgr2=0x7f30b81460b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.493+0000 7f30be729640 1 --2- 172.21.15.122:0/3889093154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b8071df0 0x7f30b81460b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f30ac01cd40 tx=0x7f30ac01ace0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.493+0000 7f30be729640 1 -- 172.21.15.122:0/3889093154 shutdown_connections 2024-03-16T20:44:58.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.493+0000 7f30be729640 1 --2- 172.21.15.122:0/3889093154 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f30a4066e40 0x7f30a4069300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.493+0000 7f30be729640 1 --2- 172.21.15.122:0/3889093154 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f30b813ecd0 0x7f30b812f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.493+0000 7f30be729640 1 --2- 172.21.15.122:0/3889093154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30b8071df0 0x7f30b81460b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.493+0000 7f30be729640 1 --2- 172.21.15.122:0/3889093154 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f30b8071a10 0x7f30b81350f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.493+0000 7f30be729640 1 -- 172.21.15.122:0/3889093154 >> 172.21.15.122:0/3889093154 conn(0x7f30b80b62a0 msgr2=0x7f30b806c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:58.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.493+0000 7f30be729640 1 -- 172.21.15.122:0/3889093154 shutdown_connections 2024-03-16T20:44:58.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.493+0000 7f30be729640 1 -- 172.21.15.122:0/3889093154 wait complete. 2024-03-16T20:44:58.515 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T20:44:52.653523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T20:44:58.516 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T20:44:58.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.629+0000 7f3b258a9640 1 Processor -- start 2024-03-16T20:44:58.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.629+0000 7f3b258a9640 1 -- start start 2024-03-16T20:44:58.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b258a9640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f3b200adf80 0x7f3b20072510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b258a9640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b200aec70 con 0x7f3b20066fa0 2024-03-16T20:44:58.637 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b258a9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b200ae690 con 0x7f3b2014b1b0 2024-03-16T20:44:58.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b258a9640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b20067380 con 0x7f3b200adf80 2024-03-16T20:44:58.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b1e7fc640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f3b200adf80 0x7f3b20072510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b1effd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f3b20066fa0 0x7f3b2014aaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:34008/0 (socket says 172.21.15.122:34008) 2024-03-16T20:44:58.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b1effd640 1 -- 172.21.15.122:0/4142236194 learned_addr learned my addr 172.21.15.122:0/4142236194 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:44:58.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b1e7fc640 1 -- 172.21.15.122:0/4142236194 >> v1:172.21.15.195:6789/0 conn(0x7f3b2014b1b0 legacy=0x7f3b20151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:58.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b1e7fc640 1 -- 172.21.15.122:0/4142236194 >> v1:172.21.15.122:6789/0 conn(0x7f3b20066fa0 legacy=0x7f3b2014aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b1e7fc640 1 -- 172.21.15.122:0/4142236194 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b200afa90 con 0x7f3b200adf80 2024-03-16T20:44:58.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b1dffb640 1 -- 172.21.15.122:0/4142236194 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1360464377 0 0) 0x7f3b200aec70 con 0x7f3b20066fa0 2024-03-16T20:44:58.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b1e7fc640 1 --2- 172.21.15.122:0/4142236194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b200adf80 0x7f3b20072510 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3b140097c0 tx=0x7f3b1401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255d529b0e3fdb6e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:58.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b1dffb640 1 -- 172.21.15.122:0/4142236194 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b140ab600 con 0x7f3b200adf80 2024-03-16T20:44:58.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b1dffb640 1 -- 172.21.15.122:0/4142236194 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b140a7050 con 0x7f3b200adf80 2024-03-16T20:44:58.640 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b1dffb640 1 -- 172.21.15.122:0/4142236194 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b140ab980 con 0x7f3b200adf80 2024-03-16T20:44:58.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b258a9640 1 -- 172.21.15.122:0/4142236194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b200adf80 msgr2=0x7f3b20072510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b258a9640 1 --2- 172.21.15.122:0/4142236194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b200adf80 0x7f3b20072510 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3b140097c0 tx=0x7f3b1401c3a0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b258a9640 1 -- 172.21.15.122:0/4142236194 shutdown_connections 2024-03-16T20:44:58.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b258a9640 1 --2- 172.21.15.122:0/4142236194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b200adf80 0x7f3b20072510 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b258a9640 1 -- 172.21.15.122:0/4142236194 >> 172.21.15.122:0/4142236194 conn(0x7f3b2006d2b0 msgr2=0x7f3b2006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:58.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b258a9640 1 -- 172.21.15.122:0/4142236194 shutdown_connections 2024-03-16T20:44:58.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.633+0000 7f3b258a9640 1 -- 172.21.15.122:0/4142236194 wait complete. 2024-03-16T20:44:58.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b258a9640 1 Processor -- start 2024-03-16T20:44:58.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b258a9640 1 -- start start 2024-03-16T20:44:58.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b258a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b20066fa0 0x7f3b20143700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b258a9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b200adf80 0x7f3b2009d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b258a9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b2014b1b0 0x7f3b200b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b258a9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2009d060 con 0x7f3b20066fa0 2024-03-16T20:44:58.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b258a9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2009e400 con 0x7f3b200adf80 2024-03-16T20:44:58.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b258a9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2009c8a0 con 0x7f3b2014b1b0 2024-03-16T20:44:58.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b1effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b20066fa0 0x7f3b20143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b1effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b20066fa0 0x7f3b20143700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33214/0 (socket says 172.21.15.122:33214) 2024-03-16T20:44:58.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b1effd640 1 -- 172.21.15.122:0/3572638882 learned_addr learned my addr 172.21.15.122:0/3572638882 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:44:58.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b1f7fe640 1 --2- 172.21.15.122:0/3572638882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b2014b1b0 0x7f3b200b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b1e7fc640 1 --2- 172.21.15.122:0/3572638882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b200adf80 0x7f3b2009d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b1effd640 1 -- 172.21.15.122:0/3572638882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b2014b1b0 msgr2=0x7f3b200b1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b1effd640 1 --2- 172.21.15.122:0/3572638882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b2014b1b0 0x7f3b200b1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b1effd640 1 -- 172.21.15.122:0/3572638882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b200adf80 msgr2=0x7f3b2009d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b1effd640 1 --2- 172.21.15.122:0/3572638882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b200adf80 0x7f3b2009d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b1effd640 1 -- 172.21.15.122:0/3572638882 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b14009420 con 0x7f3b20066fa0 2024-03-16T20:44:58.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b1f7fe640 1 --2- 172.21.15.122:0/3572638882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b2014b1b0 0x7f3b200b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:44:58.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b1effd640 1 --2- 172.21.15.122:0/3572638882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b20066fa0 0x7f3b20143700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b100011f0 tx=0x7f3b1000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:58.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b07fff640 1 -- 172.21.15.122:0/3572638882 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b100074e0 con 0x7f3b20066fa0 2024-03-16T20:44:58.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b07fff640 1 -- 172.21.15.122:0/3572638882 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b10007640 con 0x7f3b20066fa0 2024-03-16T20:44:58.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b258a9640 1 -- 172.21.15.122:0/3572638882 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b20143d60 con 0x7f3b20066fa0 2024-03-16T20:44:58.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b07fff640 1 -- 172.21.15.122:0/3572638882 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10004a10 con 0x7f3b20066fa0 2024-03-16T20:44:58.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b258a9640 1 -- 172.21.15.122:0/3572638882 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2009dcf0 con 0x7f3b20066fa0 2024-03-16T20:44:58.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.637+0000 7f3b258a9640 1 -- 172.21.15.122:0/3572638882 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b20074bb0 con 0x7f3b20066fa0 2024-03-16T20:44:58.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.641+0000 7f3b07fff640 1 -- 172.21.15.122:0/3572638882 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f3b10004b70 con 0x7f3b20066fa0 2024-03-16T20:44:58.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.641+0000 7f3b07fff640 1 --2- 172.21.15.122:0/3572638882 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3b0c066e40 0x7f3b0c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.641+0000 7f3b07fff640 1 -- 172.21.15.122:0/3572638882 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3b10083c40 con 0x7f3b20066fa0 2024-03-16T20:44:58.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.641+0000 7f3b1e7fc640 1 --2- 172.21.15.122:0/3572638882 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3b0c066e40 0x7f3b0c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.641+0000 7f3b1e7fc640 1 --2- 172.21.15.122:0/3572638882 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3b0c066e40 0x7f3b0c069300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b200b3690 tx=0x7f3b14008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:58.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.645+0000 7f3b07fff640 1 -- 172.21.15.122:0/3572638882 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3b10050e10 con 0x7f3b20066fa0 2024-03-16T20:44:58.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.793+0000 7f3b258a9640 1 -- 172.21.15.122:0/3572638882 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3b20071760 con 0x7f3b20066fa0 2024-03-16T20:44:58.800 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.793+0000 7f3b07fff640 1 -- 172.21.15.122:0/3572638882 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3b100180a0 con 0x7f3b20066fa0 2024-03-16T20:44:58.800 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-03-16T20:44:58.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.797+0000 7f3b258a9640 1 -- 172.21.15.122:0/3572638882 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3b0c066e40 msgr2=0x7f3b0c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.797+0000 7f3b258a9640 1 --2- 172.21.15.122:0/3572638882 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3b0c066e40 0x7f3b0c069300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b200b3690 tx=0x7f3b14008040 comp rx=0 tx=0).stop 2024-03-16T20:44:58.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.797+0000 7f3b258a9640 1 -- 172.21.15.122:0/3572638882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b20066fa0 msgr2=0x7f3b20143700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.797+0000 7f3b258a9640 1 --2- 172.21.15.122:0/3572638882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b20066fa0 0x7f3b20143700 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b100011f0 tx=0x7f3b1000c9b0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.797+0000 7f3b258a9640 1 -- 172.21.15.122:0/3572638882 shutdown_connections 2024-03-16T20:44:58.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.797+0000 7f3b258a9640 1 --2- 172.21.15.122:0/3572638882 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3b0c066e40 0x7f3b0c069300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.797+0000 7f3b258a9640 1 --2- 172.21.15.122:0/3572638882 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3b2014b1b0 0x7f3b200b1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.797+0000 7f3b258a9640 1 --2- 172.21.15.122:0/3572638882 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3b200adf80 0x7f3b2009d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.797+0000 7f3b258a9640 1 --2- 172.21.15.122:0/3572638882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3b20066fa0 0x7f3b20143700 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.797+0000 7f3b258a9640 1 -- 172.21.15.122:0/3572638882 >> 172.21.15.122:0/3572638882 conn(0x7f3b2006d2b0 msgr2=0x7f3b2009a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:58.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.797+0000 7f3b258a9640 1 -- 172.21.15.122:0/3572638882 shutdown_connections 2024-03-16T20:44:58.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.797+0000 7f3b258a9640 1 -- 172.21.15.122:0/3572638882 wait complete. 2024-03-16T20:44:58.817 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T20:44:58.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.917+0000 7fca98145640 1 Processor -- start 2024-03-16T20:44:58.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.917+0000 7fca98145640 1 -- start start 2024-03-16T20:44:58.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca98145640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fca90148b40 0x7fca9013f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca98145640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca900b02a0 con 0x7fca9013ed60 2024-03-16T20:44:58.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca98145640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca900b1050 con 0x7fca90072120 2024-03-16T20:44:58.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca98145640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca90072500 con 0x7fca90148b40 2024-03-16T20:44:58.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca956e0640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fca9013ed60 0x7fca90148430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:34018/0 (socket says 172.21.15.122:34018) 2024-03-16T20:44:58.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca956e0640 1 -- 172.21.15.122:0/34397779 learned_addr learned my addr 172.21.15.122:0/34397779 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:44:58.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 -- 172.21.15.122:0/34397779 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476255742 0 0) 0x7fca900b02a0 con 0x7fca9013ed60 2024-03-16T20:44:58.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 -- 172.21.15.122:0/34397779 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca7c003530 con 0x7fca9013ed60 2024-03-16T20:44:58.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca966e2640 1 --2- 172.21.15.122:0/34397779 >> v2:172.21.15.195:3300/0 conn(0x7fca90148b40 0x7fca9013f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 -- 172.21.15.122:0/34397779 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2797368892 0 0) 0x7fca7c003530 con 0x7fca9013ed60 2024-03-16T20:44:58.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 -- 172.21.15.122:0/34397779 >> v2:172.21.15.195:3300/0 conn(0x7fca90148b40 msgr2=0x7fca9013f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:58.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 --2- 172.21.15.122:0/34397779 >> v2:172.21.15.195:3300/0 conn(0x7fca90148b40 0x7fca9013f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 -- 172.21.15.122:0/34397779 >> v1:172.21.15.195:6789/0 conn(0x7fca90072120 legacy=0x7fca900afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 -- 172.21.15.122:0/34397779 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca900b38a0 con 0x7fca9013ed60 2024-03-16T20:44:58.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 -- 172.21.15.122:0/34397779 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7fca8c0016a0 con 0x7fca9013ed60 2024-03-16T20:44:58.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 -- 172.21.15.122:0/34397779 >> v1:172.21.15.122:6789/0 conn(0x7fca9013ed60 legacy=0x7fca90148430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 --2- 172.21.15.122:0/34397779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca7c003980 0x7fca7c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 --2- 172.21.15.122:0/34397779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca7c0047d0 0x7fca7c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 --2- 172.21.15.122:0/34397779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c0052b0 0x7fca7c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 -- 172.21.15.122:0/34397779 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca90072500 con 0x7fca7c0052b0 2024-03-16T20:44:58.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 -- 172.21.15.122:0/34397779 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c00c180 con 0x7fca7c0047d0 2024-03-16T20:44:58.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 -- 172.21.15.122:0/34397779 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c00c340 con 0x7fca7c003980 2024-03-16T20:44:58.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca966e2640 1 --2- 172.21.15.122:0/34397779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c0052b0 0x7fca7c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca966e2640 1 -- 172.21.15.122:0/34397779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca7c003980 msgr2=0x7fca7c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:58.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca95ee1640 1 --2- 172.21.15.122:0/34397779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca7c003980 0x7fca7c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca956e0640 1 --2- 172.21.15.122:0/34397779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca7c0047d0 0x7fca7c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca966e2640 1 --2- 172.21.15.122:0/34397779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca7c003980 0x7fca7c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca966e2640 1 -- 172.21.15.122:0/34397779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca7c0047d0 msgr2=0x7fca7c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:44:58.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca966e2640 1 --2- 172.21.15.122:0/34397779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca7c0047d0 0x7fca7c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca966e2640 1 -- 172.21.15.122:0/34397779 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c003530 con 0x7fca7c0052b0 2024-03-16T20:44:58.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca956e0640 1 --2- 172.21.15.122:0/34397779 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca7c0047d0 0x7fca7c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:44:58.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca95ee1640 1 --2- 172.21.15.122:0/34397779 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca7c003980 0x7fca7c003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:44:58.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca966e2640 1 -- 172.21.15.122:0/34397779 reap_dead start 2024-03-16T20:44:58.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca966e2640 1 --2- 172.21.15.122:0/34397779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c0052b0 0x7fca7c00b990 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fca900b3140 tx=0x7fca8801ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=496dcae1688f5b9a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:58.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 -- 172.21.15.122:0/34397779 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca880a5d20 con 0x7fca7c0052b0 2024-03-16T20:44:58.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 -- 172.21.15.122:0/34397779 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca880a5e80 con 0x7fca7c0052b0 2024-03-16T20:44:58.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.921+0000 7fca94edf640 1 -- 172.21.15.122:0/34397779 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca880ad660 con 0x7fca7c0052b0 2024-03-16T20:44:58.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 -- 172.21.15.122:0/34397779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c0052b0 msgr2=0x7fca7c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 --2- 172.21.15.122:0/34397779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c0052b0 0x7fca7c00b990 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fca900b3140 tx=0x7fca8801ce20 comp rx=0 tx=0).stop 2024-03-16T20:44:58.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 -- 172.21.15.122:0/34397779 shutdown_connections 2024-03-16T20:44:58.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 --2- 172.21.15.122:0/34397779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c0052b0 0x7fca7c00b990 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 -- 172.21.15.122:0/34397779 >> 172.21.15.122:0/34397779 conn(0x7fca900b8860 msgr2=0x7fca900b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:58.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 -- 172.21.15.122:0/34397779 shutdown_connections 2024-03-16T20:44:58.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 -- 172.21.15.122:0/34397779 wait complete. 2024-03-16T20:44:58.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 Processor -- start 2024-03-16T20:44:58.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 -- start start 2024-03-16T20:44:58.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c0052b0 0x7fca90144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca90144cc0 0x7fca901ddfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca901de510 0x7fca90143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca90145be0 con 0x7fca7c0052b0 2024-03-16T20:44:58.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca90142ff0 con 0x7fca901de510 2024-03-16T20:44:58.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca901ddab0 con 0x7fca90144cc0 2024-03-16T20:44:58.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca95ee1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c0052b0 0x7fca90144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca95ee1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c0052b0 0x7fca90144780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33240/0 (socket says 172.21.15.122:33240) 2024-03-16T20:44:58.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca95ee1640 1 -- 172.21.15.122:0/800749205 learned_addr learned my addr 172.21.15.122:0/800749205 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:44:58.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca966e2640 1 --2- 172.21.15.122:0/800749205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca901de510 0x7fca90143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca95ee1640 1 -- 172.21.15.122:0/800749205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca90144cc0 msgr2=0x7fca901ddfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca956e0640 1 --2- 172.21.15.122:0/800749205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca90144cc0 0x7fca901ddfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca95ee1640 1 --2- 172.21.15.122:0/800749205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca90144cc0 0x7fca901ddfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca95ee1640 1 -- 172.21.15.122:0/800749205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca901de510 msgr2=0x7fca90143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:58.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca95ee1640 1 --2- 172.21.15.122:0/800749205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca901de510 0x7fca90143230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:58.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca95ee1640 1 -- 172.21.15.122:0/800749205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9013f9b0 con 0x7fca7c0052b0 2024-03-16T20:44:58.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca95ee1640 1 --2- 172.21.15.122:0/800749205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c0052b0 0x7fca90144780 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fca8000d540 tx=0x7fca8000da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:58.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca867fc640 1 -- 172.21.15.122:0/800749205 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca80005480 con 0x7fca7c0052b0 2024-03-16T20:44:58.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca867fc640 1 -- 172.21.15.122:0/800749205 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca80003620 con 0x7fca7c0052b0 2024-03-16T20:44:58.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 -- 172.21.15.122:0/800749205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca900b7dd0 con 0x7fca7c0052b0 2024-03-16T20:44:58.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca867fc640 1 -- 172.21.15.122:0/800749205 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca800039d0 con 0x7fca7c0052b0 2024-03-16T20:44:58.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 -- 172.21.15.122:0/800749205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca90142c20 con 0x7fca7c0052b0 2024-03-16T20:44:58.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.925+0000 7fca98145640 1 -- 172.21.15.122:0/800749205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca901ddc70 con 0x7fca7c0052b0 2024-03-16T20:44:58.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.929+0000 7fca867fc640 1 -- 172.21.15.122:0/800749205 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fca80003b30 con 0x7fca7c0052b0 2024-03-16T20:44:58.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.929+0000 7fca867fc640 1 --2- 172.21.15.122:0/800749205 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fca74066e40 0x7fca74069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:44:58.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.929+0000 7fca867fc640 1 -- 172.21.15.122:0/800749205 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fca80084550 con 0x7fca7c0052b0 2024-03-16T20:44:58.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.929+0000 7fca956e0640 1 --2- 172.21.15.122:0/800749205 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fca74066e40 0x7fca74069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:44:58.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.929+0000 7fca956e0640 1 --2- 172.21.15.122:0/800749205 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fca74066e40 0x7fca74069300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0017d0 tx=0x7fca8c001120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:44:58.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:58.933+0000 7fca867fc640 1 -- 172.21.15.122:0/800749205 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fca80051720 con 0x7fca7c0052b0 2024-03-16T20:44:59.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:59.109+0000 7fca98145640 1 -- 172.21.15.122:0/800749205 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca90075280 con 0x7fca7c0052b0 2024-03-16T20:44:59.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:59.109+0000 7fca867fc640 1 -- 172.21.15.122:0/800749205 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7fca80054de0 con 0x7fca7c0052b0 2024-03-16T20:44:59.115 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:44:59.115 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3481092793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3481092793},{"type":"v1","addr":"172.21.15.122:6833","nonce":3481092793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-16T20:44:59.115 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 2 2024-03-16T20:44:59.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:59.109+0000 7fca98145640 1 -- 172.21.15.122:0/800749205 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fca74066e40 msgr2=0x7fca74069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:59.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:59.109+0000 7fca98145640 1 --2- 172.21.15.122:0/800749205 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fca74066e40 0x7fca74069300 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fca8c0017d0 tx=0x7fca8c001120 comp rx=0 tx=0).stop 2024-03-16T20:44:59.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:59.109+0000 7fca98145640 1 -- 172.21.15.122:0/800749205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c0052b0 msgr2=0x7fca90144780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:44:59.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:59.109+0000 7fca98145640 1 --2- 172.21.15.122:0/800749205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c0052b0 0x7fca90144780 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fca8000d540 tx=0x7fca8000da00 comp rx=0 tx=0).stop 2024-03-16T20:44:59.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:59.113+0000 7fca98145640 1 -- 172.21.15.122:0/800749205 shutdown_connections 2024-03-16T20:44:59.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:59.113+0000 7fca98145640 1 --2- 172.21.15.122:0/800749205 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fca74066e40 0x7fca74069300 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:59.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:59.113+0000 7fca98145640 1 --2- 172.21.15.122:0/800749205 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fca901de510 0x7fca90143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:59.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:59.113+0000 7fca98145640 1 --2- 172.21.15.122:0/800749205 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fca90144cc0 0x7fca901ddfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:59.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:59.113+0000 7fca98145640 1 --2- 172.21.15.122:0/800749205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fca7c0052b0 0x7fca90144780 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:44:59.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:59.113+0000 7fca98145640 1 -- 172.21.15.122:0/800749205 >> 172.21.15.122:0/800749205 conn(0x7fca900b8860 msgr2=0x7fca9013b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:44:59.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:59.113+0000 7fca98145640 1 -- 172.21.15.122:0/800749205 shutdown_connections 2024-03-16T20:44:59.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:44:59.113+0000 7fca98145640 1 -- 172.21.15.122:0/800749205 wait complete. 2024-03-16T20:45:01.132 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T20:45:01.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.257+0000 7f642b943640 1 Processor -- start 2024-03-16T20:45:01.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.261+0000 7f642b943640 1 -- start start 2024-03-16T20:45:01.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.261+0000 7f642b943640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6424147790 0x7f64240b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:01.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.261+0000 7f642b943640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6424140c70 0x7f6424141100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:01.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.261+0000 7f642b943640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64240b1390 con 0x7f64240baca0 2024-03-16T20:45:01.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.261+0000 7f642b943640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64240bb080 con 0x7f6424140c70 2024-03-16T20:45:01.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.261+0000 7f642b943640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642406d9c0 con 0x7f6424147790 2024-03-16T20:45:01.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.261+0000 7f6429ee0640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6424140c70 0x7f6424141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:01.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.261+0000 7f6429ee0640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6424140c70 0x7f6424141100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33254/0 (socket says 172.21.15.122:33254) 2024-03-16T20:45:01.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.261+0000 7f6429ee0640 1 -- 172.21.15.122:0/250475821 learned_addr learned my addr 172.21.15.122:0/250475821 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:01.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.261+0000 7f6429ee0640 1 -- 172.21.15.122:0/250475821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6424147790 msgr2=0x7f64240b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:01.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.261+0000 7f6429ee0640 1 --2- 172.21.15.122:0/250475821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6424147790 0x7f64240b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.261+0000 7f6429ee0640 1 -- 172.21.15.122:0/250475821 >> v1:172.21.15.122:6789/0 conn(0x7f64240baca0 legacy=0x7f6424147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:01.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.261+0000 7f6429ee0640 1 -- 172.21.15.122:0/250475821 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642406dca0 con 0x7f6424140c70 2024-03-16T20:45:01.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.261+0000 7f6429ee0640 1 --2- 172.21.15.122:0/250475821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6424140c70 0x7f6424141100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f641c009e50 tx=0x7f641c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c162f4d489e6d9c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:01.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.261+0000 7f6417fff640 1 -- 172.21.15.122:0/250475821 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c0a5810 con 0x7f6424140c70 2024-03-16T20:45:01.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f6417fff640 1 -- 172.21.15.122:0/250475821 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641c0a5970 con 0x7f6424140c70 2024-03-16T20:45:01.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f6417fff640 1 -- 172.21.15.122:0/250475821 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c0a5d00 con 0x7f6424140c70 2024-03-16T20:45:01.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 -- 172.21.15.122:0/250475821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6424140c70 msgr2=0x7f6424141100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:01.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 --2- 172.21.15.122:0/250475821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6424140c70 0x7f6424141100 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f641c009e50 tx=0x7f641c01ca30 comp rx=0 tx=0).stop 2024-03-16T20:45:01.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 -- 172.21.15.122:0/250475821 shutdown_connections 2024-03-16T20:45:01.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 --2- 172.21.15.122:0/250475821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6424140c70 0x7f6424141100 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 --2- 172.21.15.122:0/250475821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6424147790 0x7f64240b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 -- 172.21.15.122:0/250475821 >> 172.21.15.122:0/250475821 conn(0x7f642406ee40 msgr2=0x7f642406f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:01.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 -- 172.21.15.122:0/250475821 shutdown_connections 2024-03-16T20:45:01.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 -- 172.21.15.122:0/250475821 wait complete. 2024-03-16T20:45:01.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 Processor -- start 2024-03-16T20:45:01.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 -- start start 2024-03-16T20:45:01.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64240baca0 0x7f6424139db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:01.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6424140c70 0x7f642413f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:01.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6424147790 0x7f642413f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:01.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642413faa0 con 0x7f6424147790 2024-03-16T20:45:01.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642413e540 con 0x7f64240baca0 2024-03-16T20:45:01.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642413e820 con 0x7f6424140c70 2024-03-16T20:45:01.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f6429ee0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6424147790 0x7f642413f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:01.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f6429ee0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6424147790 0x7f642413f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33262/0 (socket says 172.21.15.122:33262) 2024-03-16T20:45:01.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f6429ee0640 1 -- 172.21.15.122:0/4021629681 learned_addr learned my addr 172.21.15.122:0/4021629681 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:01.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f64296df640 1 --2- 172.21.15.122:0/4021629681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64240baca0 0x7f6424139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:01.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f6428ede640 1 --2- 172.21.15.122:0/4021629681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6424140c70 0x7f642413f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:01.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f6429ee0640 1 -- 172.21.15.122:0/4021629681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6424140c70 msgr2=0x7f642413f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:01.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f6429ee0640 1 --2- 172.21.15.122:0/4021629681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6424140c70 0x7f642413f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f6429ee0640 1 -- 172.21.15.122:0/4021629681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64240baca0 msgr2=0x7f6424139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:01.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f6429ee0640 1 --2- 172.21.15.122:0/4021629681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64240baca0 0x7f6424139db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f6429ee0640 1 -- 172.21.15.122:0/4021629681 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641c009b30 con 0x7f6424147790 2024-03-16T20:45:01.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f6428ede640 1 --2- 172.21.15.122:0/4021629681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6424140c70 0x7f642413f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:01.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f6429ee0640 1 --2- 172.21.15.122:0/4021629681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6424147790 0x7f642413f560 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f641c01a660 tx=0x7f641c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:01.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f6415ffb640 1 -- 172.21.15.122:0/4021629681 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c0b8670 con 0x7f6424147790 2024-03-16T20:45:01.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f6415ffb640 1 -- 172.21.15.122:0/4021629681 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641c0b87d0 con 0x7f6424147790 2024-03-16T20:45:01.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 -- 172.21.15.122:0/4021629681 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642413eb00 con 0x7f6424147790 2024-03-16T20:45:01.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f6415ffb640 1 -- 172.21.15.122:0/4021629681 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641c0b8b80 con 0x7f6424147790 2024-03-16T20:45:01.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.265+0000 7f642b943640 1 -- 172.21.15.122:0/4021629681 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6424144bb0 con 0x7f6424147790 2024-03-16T20:45:01.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.269+0000 7f642b943640 1 -- 172.21.15.122:0/4021629681 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64240677b0 con 0x7f6424147790 2024-03-16T20:45:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.269+0000 7f6415ffb640 1 -- 172.21.15.122:0/4021629681 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f641c0b8ce0 con 0x7f6424147790 2024-03-16T20:45:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.269+0000 7f6415ffb640 1 --2- 172.21.15.122:0/4021629681 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6408066e40 0x7f6408069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.269+0000 7f6415ffb640 1 -- 172.21.15.122:0/4021629681 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f641c11fc50 con 0x7f6424147790 2024-03-16T20:45:01.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.269+0000 7f64296df640 1 --2- 172.21.15.122:0/4021629681 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6408066e40 0x7f6408069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:01.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.269+0000 7f64296df640 1 --2- 172.21.15.122:0/4021629681 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6408066e40 0x7f6408069300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6420001650 tx=0x7f6420009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:01.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.273+0000 7f6415ffb640 1 -- 172.21.15.122:0/4021629681 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f641c0ecf90 con 0x7f6424147790 2024-03-16T20:45:01.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.473+0000 7f642b943640 1 -- 172.21.15.122:0/4021629681 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6424075280 con 0x7f6424147790 2024-03-16T20:45:01.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.473+0000 7f6415ffb640 1 -- 172.21.15.122:0/4021629681 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2120 (secure 0 0 0) 0x7f641c0f0650 con 0x7f6424147790 2024-03-16T20:45:01.479 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:01.479 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3481092793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3481092793},{"type":"v1","addr":"172.21.15.122:6833","nonce":3481092793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4251,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/1428608180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1428608180},{"type":"v1","addr":"172.21.15.195:6835","nonce":1428608180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-03-16T20:45:01.479 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 3 2024-03-16T20:45:01.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.473+0000 7f642b943640 1 -- 172.21.15.122:0/4021629681 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6408066e40 msgr2=0x7f6408069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:01.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.473+0000 7f642b943640 1 --2- 172.21.15.122:0/4021629681 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6408066e40 0x7f6408069300 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6420001650 tx=0x7f6420009040 comp rx=0 tx=0).stop 2024-03-16T20:45:01.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.473+0000 7f642b943640 1 -- 172.21.15.122:0/4021629681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6424147790 msgr2=0x7f642413f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:01.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.473+0000 7f642b943640 1 --2- 172.21.15.122:0/4021629681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6424147790 0x7f642413f560 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f641c01a660 tx=0x7f641c0197d0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.473+0000 7f642b943640 1 -- 172.21.15.122:0/4021629681 shutdown_connections 2024-03-16T20:45:01.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.473+0000 7f642b943640 1 --2- 172.21.15.122:0/4021629681 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6408066e40 0x7f6408069300 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.473+0000 7f642b943640 1 --2- 172.21.15.122:0/4021629681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6424147790 0x7f642413f560 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.473+0000 7f642b943640 1 --2- 172.21.15.122:0/4021629681 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6424140c70 0x7f642413f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.473+0000 7f642b943640 1 --2- 172.21.15.122:0/4021629681 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64240baca0 0x7f6424139db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.473+0000 7f642b943640 1 -- 172.21.15.122:0/4021629681 >> 172.21.15.122:0/4021629681 conn(0x7f642406ee40 msgr2=0x7f6424070530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:01.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.477+0000 7f642b943640 1 -- 172.21.15.122:0/4021629681 shutdown_connections 2024-03-16T20:45:01.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.477+0000 7f642b943640 1 -- 172.21.15.122:0/4021629681 wait complete. 2024-03-16T20:45:01.497 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:01.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.613+0000 7f6fa9ab2640 1 Processor -- start 2024-03-16T20:45:01.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.613+0000 7f6fa9ab2640 1 -- start start 2024-03-16T20:45:01.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.613+0000 7f6fa9ab2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6fa413ed60 0x7f6fa413f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:01.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.613+0000 7f6fa9ab2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fa4148430 0x7f6fa41488c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:01.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.613+0000 7f6fa9ab2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fa40b02a0 con 0x7f6fa4072120 2024-03-16T20:45:01.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.613+0000 7f6fa9ab2640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fa40b1050 con 0x7f6fa413ed60 2024-03-16T20:45:01.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.613+0000 7f6fa9ab2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa4072500 con 0x7f6fa4148430 2024-03-16T20:45:01.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.617+0000 7f6fa2ffd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6fa413ed60 0x7f6fa413f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:01.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.617+0000 7f6fa3fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fa4148430 0x7f6fa41488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:01.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.617+0000 7f6fa37fe640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f6fa4072120 0x7f6fa40afb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.122:38666/0 (socket says 172.21.15.122:38666) 2024-03-16T20:45:01.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.617+0000 7f6fa3fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fa4148430 0x7f6fa41488c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:54238/0 (socket says 172.21.15.122:54238) 2024-03-16T20:45:01.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.617+0000 7f6fa2ffd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6fa413ed60 0x7f6fa413f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.122:59904/0 (socket says 172.21.15.122:59904) 2024-03-16T20:45:01.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.617+0000 7f6fa37fe640 1 -- 172.21.15.122:0/2415870442 learned_addr learned my addr 172.21.15.122:0/2415870442 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:01.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.617+0000 7f6fa3fff640 1 -- 172.21.15.122:0/2415870442 >> v2:172.21.15.195:3300/0 conn(0x7f6fa413ed60 msgr2=0x7f6fa413f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:01.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.617+0000 7f6fa3fff640 1 --2- 172.21.15.122:0/2415870442 >> v2:172.21.15.195:3300/0 conn(0x7f6fa413ed60 0x7f6fa413f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.617+0000 7f6fa3fff640 1 -- 172.21.15.122:0/2415870442 >> v1:172.21.15.195:6789/0 conn(0x7f6fa4072120 legacy=0x7f6fa40afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:01.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.617+0000 7f6fa3fff640 1 -- 172.21.15.122:0/2415870442 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa40b38a0 con 0x7f6fa4148430 2024-03-16T20:45:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.617+0000 7f6fa3fff640 1 --2- 172.21.15.122:0/2415870442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fa4148430 0x7f6fa41488c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009d90 tx=0x7f6f9c01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=406ddc67d78bf51a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.617+0000 7f6fa27fc640 1 -- 172.21.15.122:0/2415870442 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0a57a0 con 0x7f6fa4148430 2024-03-16T20:45:01.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.617+0000 7f6fa27fc640 1 -- 172.21.15.122:0/2415870442 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c0a5900 con 0x7f6fa4148430 2024-03-16T20:45:01.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.617+0000 7f6fa27fc640 1 -- 172.21.15.122:0/2415870442 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0a5c90 con 0x7f6fa4148430 2024-03-16T20:45:01.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.617+0000 7f6fa9ab2640 1 -- 172.21.15.122:0/2415870442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fa4148430 msgr2=0x7f6fa41488c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:01.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.617+0000 7f6fa9ab2640 1 --2- 172.21.15.122:0/2415870442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fa4148430 0x7f6fa41488c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009d90 tx=0x7f6f9c01c8e0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa9ab2640 1 -- 172.21.15.122:0/2415870442 shutdown_connections 2024-03-16T20:45:01.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa9ab2640 1 --2- 172.21.15.122:0/2415870442 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fa4148430 0x7f6fa41488c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa9ab2640 1 --2- 172.21.15.122:0/2415870442 >> v2:172.21.15.195:3300/0 conn(0x7f6fa413ed60 0x7f6fa413f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa9ab2640 1 -- 172.21.15.122:0/2415870442 >> 172.21.15.122:0/2415870442 conn(0x7f6fa40b8860 msgr2=0x7f6fa40b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:01.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa9ab2640 1 -- 172.21.15.122:0/2415870442 shutdown_connections 2024-03-16T20:45:01.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa9ab2640 1 -- 172.21.15.122:0/2415870442 wait complete. 2024-03-16T20:45:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa9ab2640 1 Processor -- start 2024-03-16T20:45:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa9ab2640 1 -- start start 2024-03-16T20:45:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa9ab2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fa4072120 0x7f6fa4141920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa9ab2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fa413ed60 0x7f6fa414ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa9ab2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa4148430 0x7f6fa414f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa9ab2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa41e0ed0 con 0x7f6fa4148430 2024-03-16T20:45:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa9ab2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa4142990 con 0x7f6fa413ed60 2024-03-16T20:45:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa9ab2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa4142be0 con 0x7f6fa4072120 2024-03-16T20:45:01.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa4148430 0x7f6fa414f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa4148430 0x7f6fa414f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33274/0 (socket says 172.21.15.122:33274) 2024-03-16T20:45:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa3fff640 1 -- 172.21.15.122:0/901148731 learned_addr learned my addr 172.21.15.122:0/901148731 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa37fe640 1 --2- 172.21.15.122:0/901148731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fa4072120 0x7f6fa4141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa2ffd640 1 --2- 172.21.15.122:0/901148731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fa413ed60 0x7f6fa414ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa3fff640 1 -- 172.21.15.122:0/901148731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fa4072120 msgr2=0x7f6fa4141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa3fff640 1 --2- 172.21.15.122:0/901148731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fa4072120 0x7f6fa4141920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa3fff640 1 -- 172.21.15.122:0/901148731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fa413ed60 msgr2=0x7f6fa414ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa3fff640 1 --2- 172.21.15.122:0/901148731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fa413ed60 0x7f6fa414ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa3fff640 1 -- 172.21.15.122:0/901148731 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f9c009a70 con 0x7f6fa4148430 2024-03-16T20:45:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa3fff640 1 --2- 172.21.15.122:0/901148731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa4148430 0x7f6fa414f180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009d90 tx=0x7f6f9c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6f8bfff640 1 -- 172.21.15.122:0/901148731 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0b9650 con 0x7f6fa4148430 2024-03-16T20:45:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6f8bfff640 1 -- 172.21.15.122:0/901148731 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f9c0b97b0 con 0x7f6fa4148430 2024-03-16T20:45:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6f8bfff640 1 -- 172.21.15.122:0/901148731 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c0b9b40 con 0x7f6fa4148430 2024-03-16T20:45:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa9ab2640 1 -- 172.21.15.122:0/901148731 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa41496d0 con 0x7f6fa4148430 2024-03-16T20:45:01.629 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa9ab2640 1 -- 172.21.15.122:0/901148731 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa41e13a0 con 0x7f6fa4148430 2024-03-16T20:45:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.621+0000 7f6fa9ab2640 1 -- 172.21.15.122:0/901148731 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa4075280 con 0x7f6fa4148430 2024-03-16T20:45:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.625+0000 7f6f8bfff640 1 -- 172.21.15.122:0/901148731 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f6f9c0b9ca0 con 0x7f6fa4148430 2024-03-16T20:45:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.625+0000 7f6f8bfff640 1 --2- 172.21.15.122:0/901148731 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6f90066e70 0x7f6f90069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:01.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.625+0000 7f6f8bfff640 1 -- 172.21.15.122:0/901148731 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6f9c11f8e0 con 0x7f6fa4148430 2024-03-16T20:45:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.625+0000 7f6fa37fe640 1 --2- 172.21.15.122:0/901148731 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6f90066e70 0x7f6f90069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:01.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.625+0000 7f6fa37fe640 1 --2- 172.21.15.122:0/901148731 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6f90066e70 0x7f6f90069330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6fa41480b0 tx=0x7f6f94009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:01.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.629+0000 7f6f8bfff640 1 -- 172.21.15.122:0/901148731 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6f9c0eca10 con 0x7f6fa4148430 2024-03-16T20:45:01.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.785+0000 7f6fa9ab2640 1 -- 172.21.15.122:0/901148731 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fa41493f0 con 0x7f6fa4148430 2024-03-16T20:45:01.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.789+0000 7f6f8bfff640 1 -- 172.21.15.122:0/901148731 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12920 (secure 0 0 0) 0x7f6f9c0bd090 con 0x7f6fa4148430 2024-03-16T20:45:01.794 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:01.794 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":17,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:44:55.098139+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:01.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.789+0000 7f6fa9ab2640 1 -- 172.21.15.122:0/901148731 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6f90066e70 msgr2=0x7f6f90069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:01.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.789+0000 7f6fa9ab2640 1 --2- 172.21.15.122:0/901148731 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6f90066e70 0x7f6f90069330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6fa41480b0 tx=0x7f6f94009040 comp rx=0 tx=0).stop 2024-03-16T20:45:01.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.789+0000 7f6fa9ab2640 1 -- 172.21.15.122:0/901148731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa4148430 msgr2=0x7f6fa414f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:01.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.789+0000 7f6fa9ab2640 1 --2- 172.21.15.122:0/901148731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa4148430 0x7f6fa414f180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009d90 tx=0x7f6f9c019680 comp rx=0 tx=0).stop 2024-03-16T20:45:01.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.789+0000 7f6fa9ab2640 1 -- 172.21.15.122:0/901148731 shutdown_connections 2024-03-16T20:45:01.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.789+0000 7f6fa9ab2640 1 --2- 172.21.15.122:0/901148731 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6f90066e70 0x7f6f90069330 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.789+0000 7f6fa9ab2640 1 --2- 172.21.15.122:0/901148731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6fa4148430 0x7f6fa414f180 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.789+0000 7f6fa9ab2640 1 --2- 172.21.15.122:0/901148731 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6fa413ed60 0x7f6fa414ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.789+0000 7f6fa9ab2640 1 --2- 172.21.15.122:0/901148731 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6fa4072120 0x7f6fa4141920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.789+0000 7f6fa9ab2640 1 -- 172.21.15.122:0/901148731 >> 172.21.15.122:0/901148731 conn(0x7f6fa40b8860 msgr2=0x7f6fa40b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:01.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.789+0000 7f6fa9ab2640 1 -- 172.21.15.122:0/901148731 shutdown_connections 2024-03-16T20:45:01.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.789+0000 7f6fa9ab2640 1 -- 172.21.15.122:0/901148731 wait complete. 2024-03-16T20:45:01.812 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T20:44:52.653523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T20:45:01.813 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T20:45:01.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.925+0000 7f6062ee0640 1 Processor -- start 2024-03-16T20:45:01.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.925+0000 7f6062ee0640 1 -- start start 2024-03-16T20:45:01.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f6062ee0640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f605c09ae30 0x7f605c0b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:01.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f6062ee0640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f605c09b210 con 0x7f605c144aa0 2024-03-16T20:45:01.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f6062ee0640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f605c0b3320 con 0x7f605c0aef80 2024-03-16T20:45:01.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f6062ee0640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f605c0b3110 con 0x7f605c09ae30 2024-03-16T20:45:01.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f6060c7c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f605c09ae30 0x7f605c0b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:01.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f6060c7c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f605c09ae30 0x7f605c0b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33290/0 (socket says 172.21.15.122:33290) 2024-03-16T20:45:01.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f606147d640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f605c144aa0 0x7f605c14e850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:34030/0 (socket says 172.21.15.122:34030) 2024-03-16T20:45:01.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f606147d640 1 -- 172.21.15.122:0/2108123243 learned_addr learned my addr 172.21.15.122:0/2108123243 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:01.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f6060c7c640 1 -- 172.21.15.122:0/2108123243 >> v1:172.21.15.195:6789/0 conn(0x7f605c0aef80 legacy=0x7f605c144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:01.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f6060c7c640 1 -- 172.21.15.122:0/2108123243 >> v1:172.21.15.122:6789/0 conn(0x7f605c144aa0 legacy=0x7f605c14e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:01.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f6060c7c640 1 -- 172.21.15.122:0/2108123243 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c0b3600 con 0x7f605c09ae30 2024-03-16T20:45:01.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f605b7fe640 1 -- 172.21.15.122:0/2108123243 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473273829 0 0) 0x7f605c09b210 con 0x7f605c144aa0 2024-03-16T20:45:01.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f6060c7c640 1 --2- 172.21.15.122:0/2108123243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f605c09ae30 0x7f605c0b3b60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f605c0b3250 tx=0x7f604c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba307ae320e8bc70 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:01.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f605b7fe640 1 -- 172.21.15.122:0/2108123243 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c0a4ea0 con 0x7f605c09ae30 2024-03-16T20:45:01.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f605b7fe640 1 -- 172.21.15.122:0/2108123243 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604c0a7050 con 0x7f605c09ae30 2024-03-16T20:45:01.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f605b7fe640 1 -- 172.21.15.122:0/2108123243 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c0ab820 con 0x7f605c09ae30 2024-03-16T20:45:01.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f6062ee0640 1 -- 172.21.15.122:0/2108123243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f605c09ae30 msgr2=0x7f605c0b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:01.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f6062ee0640 1 --2- 172.21.15.122:0/2108123243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f605c09ae30 0x7f605c0b3b60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f605c0b3250 tx=0x7f604c01c410 comp rx=0 tx=0).stop 2024-03-16T20:45:01.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f6062ee0640 1 -- 172.21.15.122:0/2108123243 shutdown_connections 2024-03-16T20:45:01.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f6062ee0640 1 --2- 172.21.15.122:0/2108123243 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f605c09ae30 0x7f605c0b3b60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.929+0000 7f6062ee0640 1 -- 172.21.15.122:0/2108123243 >> 172.21.15.122:0/2108123243 conn(0x7f605c0b8d10 msgr2=0x7f605c09a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:01.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f6062ee0640 1 -- 172.21.15.122:0/2108123243 shutdown_connections 2024-03-16T20:45:01.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f6062ee0640 1 -- 172.21.15.122:0/2108123243 wait complete. 2024-03-16T20:45:01.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f6062ee0640 1 Processor -- start 2024-03-16T20:45:01.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f6062ee0640 1 -- start start 2024-03-16T20:45:01.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f6062ee0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f605c09ae30 0x7f605c1e0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:01.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f6062ee0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f605c0aef80 0x7f605c1df3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:01.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f6062ee0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f605c144aa0 0x7f605c1df910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:01.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f6062ee0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c1e1b50 con 0x7f605c0aef80 2024-03-16T20:45:01.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f6062ee0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f605c141c20 con 0x7f605c144aa0 2024-03-16T20:45:01.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f6062ee0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f605c1deeb0 con 0x7f605c09ae30 2024-03-16T20:45:01.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f605bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f605c0aef80 0x7f605c1df3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:01.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f605bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f605c0aef80 0x7f605c1df3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33306/0 (socket says 172.21.15.122:33306) 2024-03-16T20:45:01.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f605bfff640 1 -- 172.21.15.122:0/1975198548 learned_addr learned my addr 172.21.15.122:0/1975198548 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:01.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f6060c7c640 1 --2- 172.21.15.122:0/1975198548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f605c09ae30 0x7f605c1e0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:01.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f606147d640 1 --2- 172.21.15.122:0/1975198548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f605c144aa0 0x7f605c1df910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:01.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f605bfff640 1 -- 172.21.15.122:0/1975198548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f605c09ae30 msgr2=0x7f605c1e0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:01.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f605bfff640 1 --2- 172.21.15.122:0/1975198548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f605c09ae30 0x7f605c1e0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f605bfff640 1 -- 172.21.15.122:0/1975198548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f605c144aa0 msgr2=0x7f605c1df910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:01.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f605bfff640 1 --2- 172.21.15.122:0/1975198548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f605c144aa0 0x7f605c1df910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:01.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f605bfff640 1 -- 172.21.15.122:0/1975198548 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f604c009420 con 0x7f605c0aef80 2024-03-16T20:45:01.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f606147d640 1 --2- 172.21.15.122:0/1975198548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f605c144aa0 0x7f605c1df910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:01.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f605bfff640 1 --2- 172.21.15.122:0/1975198548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f605c0aef80 0x7f605c1df3d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f605000c500 tx=0x7f605000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:01.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f60597fa640 1 -- 172.21.15.122:0/1975198548 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6050005220 con 0x7f605c0aef80 2024-03-16T20:45:01.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f60597fa640 1 -- 172.21.15.122:0/1975198548 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6050005380 con 0x7f605c0aef80 2024-03-16T20:45:01.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f6062ee0640 1 -- 172.21.15.122:0/1975198548 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605c141940 con 0x7f605c0aef80 2024-03-16T20:45:01.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f60597fa640 1 -- 172.21.15.122:0/1975198548 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6050007270 con 0x7f605c0aef80 2024-03-16T20:45:01.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f6062ee0640 1 -- 172.21.15.122:0/1975198548 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605c0ae580 con 0x7f605c0aef80 2024-03-16T20:45:01.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.933+0000 7f6062ee0640 1 -- 172.21.15.122:0/1975198548 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f605c1e1cf0 con 0x7f605c0aef80 2024-03-16T20:45:01.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.937+0000 7f60597fa640 1 -- 172.21.15.122:0/1975198548 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f60500073d0 con 0x7f605c0aef80 2024-03-16T20:45:01.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.937+0000 7f60597fa640 1 --2- 172.21.15.122:0/1975198548 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6048066e70 0x7f6048069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:01.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.937+0000 7f60597fa640 1 -- 172.21.15.122:0/1975198548 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6050083970 con 0x7f605c0aef80 2024-03-16T20:45:01.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.937+0000 7f6060c7c640 1 --2- 172.21.15.122:0/1975198548 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6048066e70 0x7f6048069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:01.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.937+0000 7f6060c7c640 1 --2- 172.21.15.122:0/1975198548 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6048066e70 0x7f6048069330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f604c0ad000 tx=0x7f604c0a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:01.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:01.941+0000 7f60597fa640 1 -- 172.21.15.122:0/1975198548 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6050050b70 con 0x7f605c0aef80 2024-03-16T20:45:02.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.089+0000 7f6062ee0640 1 -- 172.21.15.122:0/1975198548 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f605c074bb0 con 0x7f605c0aef80 2024-03-16T20:45:02.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.089+0000 7f60597fa640 1 -- 172.21.15.122:0/1975198548 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6050054230 con 0x7f605c0aef80 2024-03-16T20:45:02.097 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-03-16T20:45:02.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.093+0000 7f6062ee0640 1 -- 172.21.15.122:0/1975198548 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6048066e70 msgr2=0x7f6048069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:02.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.093+0000 7f6062ee0640 1 --2- 172.21.15.122:0/1975198548 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6048066e70 0x7f6048069330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f604c0ad000 tx=0x7f604c0a6000 comp rx=0 tx=0).stop 2024-03-16T20:45:02.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.093+0000 7f6062ee0640 1 -- 172.21.15.122:0/1975198548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f605c0aef80 msgr2=0x7f605c1df3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:02.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.093+0000 7f6062ee0640 1 --2- 172.21.15.122:0/1975198548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f605c0aef80 0x7f605c1df3d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f605000c500 tx=0x7f605000c9c0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.093+0000 7f6062ee0640 1 -- 172.21.15.122:0/1975198548 shutdown_connections 2024-03-16T20:45:02.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.093+0000 7f6062ee0640 1 --2- 172.21.15.122:0/1975198548 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6048066e70 0x7f6048069330 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.093+0000 7f6062ee0640 1 --2- 172.21.15.122:0/1975198548 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f605c144aa0 0x7f605c1df910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.093+0000 7f6062ee0640 1 --2- 172.21.15.122:0/1975198548 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f605c0aef80 0x7f605c1df3d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.093+0000 7f6062ee0640 1 --2- 172.21.15.122:0/1975198548 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f605c09ae30 0x7f605c1e0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.093+0000 7f6062ee0640 1 -- 172.21.15.122:0/1975198548 >> 172.21.15.122:0/1975198548 conn(0x7f605c0b8d10 msgr2=0x7f605c0add40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:02.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.093+0000 7f6062ee0640 1 -- 172.21.15.122:0/1975198548 shutdown_connections 2024-03-16T20:45:02.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.093+0000 7f6062ee0640 1 -- 172.21.15.122:0/1975198548 wait complete. 2024-03-16T20:45:02.113 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-16T20:45:02.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.225+0000 7f3426963640 1 Processor -- start 2024-03-16T20:45:02.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.225+0000 7f3426963640 1 -- start start 2024-03-16T20:45:02.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.225+0000 7f3426963640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3420144290 0x7f3420144670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:02.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.225+0000 7f3426963640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3420144bb0 0x7f342013de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:02.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.225+0000 7f3426963640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f342006ec30 con 0x7f342013e3c0 2024-03-16T20:45:02.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.225+0000 7f3426963640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34200b1260 con 0x7f3420144290 2024-03-16T20:45:02.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.225+0000 7f3426963640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34200b1440 con 0x7f3420144bb0 2024-03-16T20:45:02.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.225+0000 7f34177fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3420144bb0 0x7f342013de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:02.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.225+0000 7f341ffff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3420144290 0x7f3420144670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:02.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.225+0000 7f34177fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3420144bb0 0x7f342013de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:54264/0 (socket says 172.21.15.122:54264) 2024-03-16T20:45:02.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.225+0000 7f34177fe640 1 -- 172.21.15.122:0/1567240689 learned_addr learned my addr 172.21.15.122:0/1567240689 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:02.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.225+0000 7f34177fe640 1 -- 172.21.15.122:0/1567240689 >> v2:172.21.15.195:3300/0 conn(0x7f3420144290 msgr2=0x7f3420144670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:02.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.225+0000 7f34177fe640 1 --2- 172.21.15.122:0/1567240689 >> v2:172.21.15.195:3300/0 conn(0x7f3420144290 0x7f3420144670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.225+0000 7f34177fe640 1 -- 172.21.15.122:0/1567240689 >> v1:172.21.15.195:6789/0 conn(0x7f342013e3c0 legacy=0x7f3420147530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:02.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.225+0000 7f34177fe640 1 -- 172.21.15.122:0/1567240689 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f342006e950 con 0x7f3420144bb0 2024-03-16T20:45:02.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f34177fe640 1 --2- 172.21.15.122:0/1567240689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3420144bb0 0x7f342013de80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3408009870 tx=0x7f340801c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecd3130e77c7b2c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:02.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f341f7fe640 1 -- 172.21.15.122:0/1567240689 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34080ab600 con 0x7f3420144bb0 2024-03-16T20:45:02.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f341f7fe640 1 -- 172.21.15.122:0/1567240689 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34080a7050 con 0x7f3420144bb0 2024-03-16T20:45:02.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f341f7fe640 1 -- 172.21.15.122:0/1567240689 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34080ab980 con 0x7f3420144bb0 2024-03-16T20:45:02.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f3426963640 1 -- 172.21.15.122:0/1567240689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3420144bb0 msgr2=0x7f342013de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:02.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f3426963640 1 --2- 172.21.15.122:0/1567240689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3420144bb0 0x7f342013de80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3408009870 tx=0x7f340801c440 comp rx=0 tx=0).stop 2024-03-16T20:45:02.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f3426963640 1 -- 172.21.15.122:0/1567240689 shutdown_connections 2024-03-16T20:45:02.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f3426963640 1 --2- 172.21.15.122:0/1567240689 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3420144bb0 0x7f342013de80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f3426963640 1 --2- 172.21.15.122:0/1567240689 >> v2:172.21.15.195:3300/0 conn(0x7f3420144290 0x7f3420144670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f3426963640 1 -- 172.21.15.122:0/1567240689 >> 172.21.15.122:0/1567240689 conn(0x7f3420070d30 msgr2=0x7f3420071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:02.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f3426963640 1 -- 172.21.15.122:0/1567240689 shutdown_connections 2024-03-16T20:45:02.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f3426963640 1 -- 172.21.15.122:0/1567240689 wait complete. 2024-03-16T20:45:02.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f3426963640 1 Processor -- start 2024-03-16T20:45:02.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f3426963640 1 -- start start 2024-03-16T20:45:02.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f3426963640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f342013e3c0 0x7f34201316a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:02.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f3426963640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3420144290 0x7f34200b1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:02.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f3426963640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3420144bb0 0x7f3420136650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:02.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f3426963640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34201361c0 con 0x7f3420144290 2024-03-16T20:45:02.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f3426963640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3420135bb0 con 0x7f342013e3c0 2024-03-16T20:45:02.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.229+0000 7f3426963640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34200b2f00 con 0x7f3420144bb0 2024-03-16T20:45:02.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f34177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3420144290 0x7f34200b1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:02.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f34177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3420144290 0x7f34200b1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33308/0 (socket says 172.21.15.122:33308) 2024-03-16T20:45:02.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f34177fe640 1 -- 172.21.15.122:0/2652956074 learned_addr learned my addr 172.21.15.122:0/2652956074 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:02.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f3424f00640 1 --2- 172.21.15.122:0/2652956074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3420144bb0 0x7f3420136650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:02.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f341ffff640 1 --2- 172.21.15.122:0/2652956074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f342013e3c0 0x7f34201316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:02.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f34177fe640 1 -- 172.21.15.122:0/2652956074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3420144bb0 msgr2=0x7f3420136650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:02.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f34177fe640 1 --2- 172.21.15.122:0/2652956074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3420144bb0 0x7f3420136650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f34177fe640 1 -- 172.21.15.122:0/2652956074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f342013e3c0 msgr2=0x7f34201316a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:02.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f34177fe640 1 --2- 172.21.15.122:0/2652956074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f342013e3c0 0x7f34201316a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f34177fe640 1 -- 172.21.15.122:0/2652956074 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34080094d0 con 0x7f3420144290 2024-03-16T20:45:02.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f3424f00640 1 --2- 172.21.15.122:0/2652956074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3420144bb0 0x7f3420136650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:02.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f34177fe640 1 --2- 172.21.15.122:0/2652956074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3420144290 0x7f34200b1cc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f34080ae040 tx=0x7f34080191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:02.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f341d7fa640 1 -- 172.21.15.122:0/2652956074 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34080abd30 con 0x7f3420144290 2024-03-16T20:45:02.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f341d7fa640 1 -- 172.21.15.122:0/2652956074 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34080abe90 con 0x7f3420144290 2024-03-16T20:45:02.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f3426963640 1 -- 172.21.15.122:0/2652956074 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34200b76d0 con 0x7f3420144290 2024-03-16T20:45:02.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f341d7fa640 1 -- 172.21.15.122:0/2652956074 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34080b4680 con 0x7f3420144290 2024-03-16T20:45:02.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f3426963640 1 -- 172.21.15.122:0/2652956074 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34200b2b80 con 0x7f3420144290 2024-03-16T20:45:02.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f3426963640 1 -- 172.21.15.122:0/2652956074 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34200ba6b0 con 0x7f3420144290 2024-03-16T20:45:02.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f341d7fa640 1 -- 172.21.15.122:0/2652956074 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f34080b2070 con 0x7f3420144290 2024-03-16T20:45:02.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f341d7fa640 1 --2- 172.21.15.122:0/2652956074 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3400066e40 0x7f3400069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:02.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f341d7fa640 1 -- 172.21.15.122:0/2652956074 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f340811e3f0 con 0x7f3420144290 2024-03-16T20:45:02.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f341ffff640 1 --2- 172.21.15.122:0/2652956074 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3400066e40 0x7f3400069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:02.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.233+0000 7f341ffff640 1 --2- 172.21.15.122:0/2652956074 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3400066e40 0x7f3400069300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3410009a50 tx=0x7f3410008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:02.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.237+0000 7f341d7fa640 1 -- 172.21.15.122:0/2652956074 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f34080eb540 con 0x7f3420144290 2024-03-16T20:45:02.404 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.397+0000 7f3426963640 1 -- 172.21.15.122:0/2652956074 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3420075280 con 0x7f3420144290 2024-03-16T20:45:02.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.401+0000 7f341d7fa640 1 -- 172.21.15.122:0/2652956074 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f34080eec00 con 0x7f3420144290 2024-03-16T20:45:02.405 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:02.405 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-03-16T20:45:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: "poolnum": 1, 2024-03-16T20:45:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: "poolname": ".mgr" 2024-03-16T20:45:02.406 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:02.406 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-03-16T20:45:02.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.401+0000 7f3426963640 1 -- 172.21.15.122:0/2652956074 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3400066e40 msgr2=0x7f3400069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:02.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.401+0000 7f3426963640 1 --2- 172.21.15.122:0/2652956074 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3400066e40 0x7f3400069300 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3410009a50 tx=0x7f3410008040 comp rx=0 tx=0).stop 2024-03-16T20:45:02.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.401+0000 7f3426963640 1 -- 172.21.15.122:0/2652956074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3420144290 msgr2=0x7f34200b1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:02.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.401+0000 7f3426963640 1 --2- 172.21.15.122:0/2652956074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3420144290 0x7f34200b1cc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f34080ae040 tx=0x7f34080191e0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.401+0000 7f3426963640 1 -- 172.21.15.122:0/2652956074 shutdown_connections 2024-03-16T20:45:02.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.401+0000 7f3426963640 1 --2- 172.21.15.122:0/2652956074 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3400066e40 0x7f3400069300 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.401+0000 7f3426963640 1 --2- 172.21.15.122:0/2652956074 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3420144bb0 0x7f3420136650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.401+0000 7f3426963640 1 --2- 172.21.15.122:0/2652956074 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3420144290 0x7f34200b1cc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.401+0000 7f3426963640 1 --2- 172.21.15.122:0/2652956074 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f342013e3c0 0x7f34201316a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.401+0000 7f3426963640 1 -- 172.21.15.122:0/2652956074 >> 172.21.15.122:0/2652956074 conn(0x7f3420070d30 msgr2=0x7f3420071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:02.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.401+0000 7f3426963640 1 -- 172.21.15.122:0/2652956074 shutdown_connections 2024-03-16T20:45:02.408 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.401+0000 7f3426963640 1 -- 172.21.15.122:0/2652956074 wait complete. 2024-03-16T20:45:02.424 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-16T20:45:02.424 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-16T20:45:02.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.545+0000 7f5e43fc0640 1 Processor -- start 2024-03-16T20:45:02.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.545+0000 7f5e43fc0640 1 -- start start 2024-03-16T20:45:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.545+0000 7f5e43fc0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5e3c09ae30 0x7f5e3c0b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.545+0000 7f5e43fc0640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5e3c1429f0 0x7f5e3c0b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.545+0000 7f5e43fc0640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3c0b3fa0 con 0x7f5e3c0b7ba0 2024-03-16T20:45:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.545+0000 7f5e43fc0640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c0b50a0 con 0x7f5e3c1429f0 2024-03-16T20:45:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.545+0000 7f5e43fc0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c0b8590 con 0x7f5e3c09ae30 2024-03-16T20:45:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.545+0000 7f5e4155b640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f5e3c0b7ba0 0x7f5e3c1422e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:34038/0 (socket says 172.21.15.122:34038) 2024-03-16T20:45:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.545+0000 7f5e4155b640 1 -- 172.21.15.122:0/3229325849 learned_addr learned my addr 172.21.15.122:0/3229325849 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.545+0000 7f5e4255d640 1 --2- 172.21.15.122:0/3229325849 >> v2:172.21.15.122:3300/0 conn(0x7f5e3c1429f0 0x7f5e3c0b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.545+0000 7f5e41d5c640 1 --2- 172.21.15.122:0/3229325849 >> v2:172.21.15.195:3300/0 conn(0x7f5e3c09ae30 0x7f5e3c0b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.545+0000 7f5e4255d640 1 -- 172.21.15.122:0/3229325849 >> v2:172.21.15.195:3300/0 conn(0x7f5e3c09ae30 msgr2=0x7f5e3c0b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.545+0000 7f5e4255d640 1 --2- 172.21.15.122:0/3229325849 >> v2:172.21.15.195:3300/0 conn(0x7f5e3c09ae30 0x7f5e3c0b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.545+0000 7f5e4255d640 1 -- 172.21.15.122:0/3229325849 >> v1:172.21.15.122:6789/0 conn(0x7f5e3c0b7ba0 legacy=0x7f5e3c1422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.545+0000 7f5e4255d640 1 -- 172.21.15.122:0/3229325849 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c09b210 con 0x7f5e3c1429f0 2024-03-16T20:45:02.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.545+0000 7f5e40d5a640 1 -- 172.21.15.122:0/3229325849 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1447784321 0 0) 0x7f5e3c0b3fa0 con 0x7f5e3c0b7ba0 2024-03-16T20:45:02.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e4255d640 1 --2- 172.21.15.122:0/3229325849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c1429f0 0x7f5e3c0b8050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c06e120 tx=0x7f5e3401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e6af516c68dc6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:02.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e40d5a640 1 -- 172.21.15.122:0/3229325849 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e340a59f0 con 0x7f5e3c1429f0 2024-03-16T20:45:02.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e40d5a640 1 -- 172.21.15.122:0/3229325849 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e340a5b50 con 0x7f5e3c1429f0 2024-03-16T20:45:02.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e40d5a640 1 -- 172.21.15.122:0/3229325849 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e340ad440 con 0x7f5e3c1429f0 2024-03-16T20:45:02.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e43fc0640 1 -- 172.21.15.122:0/3229325849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c1429f0 msgr2=0x7f5e3c0b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:02.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e43fc0640 1 --2- 172.21.15.122:0/3229325849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c1429f0 0x7f5e3c0b8050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c06e120 tx=0x7f5e3401c930 comp rx=0 tx=0).stop 2024-03-16T20:45:02.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e43fc0640 1 -- 172.21.15.122:0/3229325849 shutdown_connections 2024-03-16T20:45:02.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e43fc0640 1 --2- 172.21.15.122:0/3229325849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c1429f0 0x7f5e3c0b8050 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e43fc0640 1 --2- 172.21.15.122:0/3229325849 >> v2:172.21.15.195:3300/0 conn(0x7f5e3c09ae30 0x7f5e3c0b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e43fc0640 1 -- 172.21.15.122:0/3229325849 >> 172.21.15.122:0/3229325849 conn(0x7f5e3c0b8d10 msgr2=0x7f5e3c09a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e43fc0640 1 -- 172.21.15.122:0/3229325849 shutdown_connections 2024-03-16T20:45:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e43fc0640 1 -- 172.21.15.122:0/3229325849 wait complete. 2024-03-16T20:45:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e43fc0640 1 Processor -- start 2024-03-16T20:45:02.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e43fc0640 1 -- start start 2024-03-16T20:45:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e43fc0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c09ae30 0x7f5e3c1459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e43fc0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e3c0b7ba0 0x7f5e3c0b1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e43fc0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e3c1429f0 0x7f5e3c0b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e43fc0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0b0270 con 0x7f5e3c09ae30 2024-03-16T20:45:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e43fc0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c144c60 con 0x7f5e3c0b7ba0 2024-03-16T20:45:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e43fc0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0b1dc0 con 0x7f5e3c1429f0 2024-03-16T20:45:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e41d5c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c09ae30 0x7f5e3c1459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:02.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e41d5c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c09ae30 0x7f5e3c1459e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33320/0 (socket says 172.21.15.122:33320) 2024-03-16T20:45:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e41d5c640 1 -- 172.21.15.122:0/2732274343 learned_addr learned my addr 172.21.15.122:0/2732274343 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e4255d640 1 --2- 172.21.15.122:0/2732274343 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e3c1429f0 0x7f5e3c0b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e41d5c640 1 -- 172.21.15.122:0/2732274343 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e3c1429f0 msgr2=0x7f5e3c0b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e41d5c640 1 --2- 172.21.15.122:0/2732274343 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e3c1429f0 0x7f5e3c0b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e4155b640 1 --2- 172.21.15.122:0/2732274343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e3c0b7ba0 0x7f5e3c0b1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e41d5c640 1 -- 172.21.15.122:0/2732274343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e3c0b7ba0 msgr2=0x7f5e3c0b1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e41d5c640 1 --2- 172.21.15.122:0/2732274343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e3c0b7ba0 0x7f5e3c0b1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e41d5c640 1 -- 172.21.15.122:0/2732274343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e34009a70 con 0x7f5e3c09ae30 2024-03-16T20:45:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e4255d640 1 --2- 172.21.15.122:0/2732274343 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e3c1429f0 0x7f5e3c0b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e41d5c640 1 --2- 172.21.15.122:0/2732274343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c09ae30 0x7f5e3c1459e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00d800 tx=0x7f5e2c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e327fc640 1 -- 172.21.15.122:0/2732274343 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00b6c0 con 0x7f5e3c09ae30 2024-03-16T20:45:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e327fc640 1 -- 172.21.15.122:0/2732274343 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c00b820 con 0x7f5e3c09ae30 2024-03-16T20:45:02.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e43fc0640 1 -- 172.21.15.122:0/2732274343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c0b20a0 con 0x7f5e3c09ae30 2024-03-16T20:45:02.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.549+0000 7f5e327fc640 1 -- 172.21.15.122:0/2732274343 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00bb90 con 0x7f5e3c09ae30 2024-03-16T20:45:02.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.553+0000 7f5e43fc0640 1 -- 172.21.15.122:0/2732274343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c0b1860 con 0x7f5e3c09ae30 2024-03-16T20:45:02.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.553+0000 7f5e43fc0640 1 -- 172.21.15.122:0/2732274343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c0684a0 con 0x7f5e3c09ae30 2024-03-16T20:45:02.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.553+0000 7f5e327fc640 1 -- 172.21.15.122:0/2732274343 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f5e2c00bcf0 con 0x7f5e3c09ae30 2024-03-16T20:45:02.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.553+0000 7f5e327fc640 1 --2- 172.21.15.122:0/2732274343 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5e20066e70 0x7f5e20069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:02.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.553+0000 7f5e327fc640 1 -- 172.21.15.122:0/2732274343 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5e2c084bc0 con 0x7f5e3c09ae30 2024-03-16T20:45:02.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.553+0000 7f5e4155b640 1 --2- 172.21.15.122:0/2732274343 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5e20066e70 0x7f5e20069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:02.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.553+0000 7f5e4155b640 1 --2- 172.21.15.122:0/2732274343 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5e20066e70 0x7f5e20069330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e380011f0 tx=0x7f5e38009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:02.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.557+0000 7f5e327fc640 1 -- 172.21.15.122:0/2732274343 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5e2c051d90 con 0x7f5e3c09ae30 2024-03-16T20:45:02.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:02.717+0000 7f5e43fc0640 1 -- 172.21.15.122:0/2732274343 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5e3c074bb0 con 0x7f5e3c09ae30 2024-03-16T20:45:03.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.333+0000 7f5e327fc640 1 -- 172.21.15.122:0/2732274343 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5e2c055450 con 0x7f5e3c09ae30 2024-03-16T20:45:03.340 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' created 2024-03-16T20:45:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.337+0000 7f5e43fc0640 1 -- 172.21.15.122:0/2732274343 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5e20066e70 msgr2=0x7f5e20069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.337+0000 7f5e43fc0640 1 --2- 172.21.15.122:0/2732274343 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5e20066e70 0x7f5e20069330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e380011f0 tx=0x7f5e38009040 comp rx=0 tx=0).stop 2024-03-16T20:45:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.337+0000 7f5e43fc0640 1 -- 172.21.15.122:0/2732274343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c09ae30 msgr2=0x7f5e3c1459e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.337+0000 7f5e43fc0640 1 --2- 172.21.15.122:0/2732274343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c09ae30 0x7f5e3c1459e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00d800 tx=0x7f5e2c00dcc0 comp rx=0 tx=0).stop 2024-03-16T20:45:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.337+0000 7f5e43fc0640 1 -- 172.21.15.122:0/2732274343 shutdown_connections 2024-03-16T20:45:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.337+0000 7f5e43fc0640 1 --2- 172.21.15.122:0/2732274343 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5e20066e70 0x7f5e20069330 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e380011f0 tx=0x7f5e38009040 comp rx=0 tx=0).stop 2024-03-16T20:45:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.337+0000 7f5e43fc0640 1 --2- 172.21.15.122:0/2732274343 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e3c1429f0 0x7f5e3c0b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.337+0000 7f5e43fc0640 1 --2- 172.21.15.122:0/2732274343 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e3c0b7ba0 0x7f5e3c0b1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.337+0000 7f5e43fc0640 1 --2- 172.21.15.122:0/2732274343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c09ae30 0x7f5e3c1459e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:03.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.337+0000 7f5e43fc0640 1 -- 172.21.15.122:0/2732274343 >> 172.21.15.122:0/2732274343 conn(0x7f5e3c0b8d10 msgr2=0x7f5e3c139f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:03.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.337+0000 7f5e43fc0640 1 -- 172.21.15.122:0/2732274343 shutdown_connections 2024-03-16T20:45:03.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.341+0000 7f5e43fc0640 1 -- 172.21.15.122:0/2732274343 wait complete. 2024-03-16T20:45:03.371 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-16T20:45:03.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64d1cf8640 1 Processor -- start 2024-03-16T20:45:03.499 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64d1cf8640 1 -- start start 2024-03-16T20:45:03.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64d1cf8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f64cc0bb1d0 0x7f64cc06cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:03.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64d1cf8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64cc06d180 0x7f64cc151920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:03.502 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64d1cf8640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64cc147ff0 con 0x7f64cc141580 2024-03-16T20:45:03.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64d1cf8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64cc1479a0 con 0x7f64cc0bb1d0 2024-03-16T20:45:03.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64d1cf8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc147c80 con 0x7f64cc06d180 2024-03-16T20:45:03.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64cb7fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f64cc141580 0x7f64cc0baac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:34048/0 (socket says 172.21.15.122:34048) 2024-03-16T20:45:03.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64cb7fe640 1 -- 172.21.15.122:0/2451563908 learned_addr learned my addr 172.21.15.122:0/2451563908 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:03.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64ca7fc640 1 -- 172.21.15.122:0/2451563908 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547121522 0 0) 0x7f64cc147ff0 con 0x7f64cc141580 2024-03-16T20:45:03.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64ca7fc640 1 -- 172.21.15.122:0/2451563908 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64b8003530 con 0x7f64cc141580 2024-03-16T20:45:03.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64caffd640 1 --2- 172.21.15.122:0/2451563908 >> v2:172.21.15.195:3300/0 conn(0x7f64cc0bb1d0 0x7f64cc06cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:03.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64cbfff640 1 --2- 172.21.15.122:0/2451563908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64cc06d180 0x7f64cc151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:03.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64ca7fc640 1 -- 172.21.15.122:0/2451563908 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2246648617 0 0) 0x7f64b8003530 con 0x7f64cc141580 2024-03-16T20:45:03.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64ca7fc640 1 -- 172.21.15.122:0/2451563908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64cc06d180 msgr2=0x7f64cc151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:03.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64ca7fc640 1 --2- 172.21.15.122:0/2451563908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64cc06d180 0x7f64cc151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:03.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64ca7fc640 1 -- 172.21.15.122:0/2451563908 >> v2:172.21.15.195:3300/0 conn(0x7f64cc0bb1d0 msgr2=0x7f64cc06cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:03.503 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64ca7fc640 1 --2- 172.21.15.122:0/2451563908 >> v2:172.21.15.195:3300/0 conn(0x7f64cc0bb1d0 0x7f64cc06cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:03.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64ca7fc640 1 -- 172.21.15.122:0/2451563908 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc1475b0 con 0x7f64cc141580 2024-03-16T20:45:03.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64ca7fc640 1 -- 172.21.15.122:0/2451563908 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f64c4001680 con 0x7f64cc141580 2024-03-16T20:45:03.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64ca7fc640 1 -- 172.21.15.122:0/2451563908 >> v1:172.21.15.122:6789/0 conn(0x7f64cc141580 legacy=0x7f64cc0baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:03.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64ca7fc640 1 --2- 172.21.15.122:0/2451563908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64b8003980 0x7f64b8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:03.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64ca7fc640 1 --2- 172.21.15.122:0/2451563908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64cc06d180 0x7f64b80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:03.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64ca7fc640 1 --2- 172.21.15.122:0/2451563908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64b8004d10 0x7f64b8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:03.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64ca7fc640 1 -- 172.21.15.122:0/2451563908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc1479a0 con 0x7f64b8004d10 2024-03-16T20:45:03.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64ca7fc640 1 -- 172.21.15.122:0/2451563908 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc147c80 con 0x7f64b8003980 2024-03-16T20:45:03.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.493+0000 7f64ca7fc640 1 -- 172.21.15.122:0/2451563908 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b800ba40 con 0x7f64cc06d180 2024-03-16T20:45:03.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64cbfff640 1 --2- 172.21.15.122:0/2451563908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64b8004d10 0x7f64b8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:03.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64cb7fe640 1 --2- 172.21.15.122:0/2451563908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64b8003980 0x7f64b8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:03.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64caffd640 1 --2- 172.21.15.122:0/2451563908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64cc06d180 0x7f64b80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:03.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64cbfff640 1 -- 172.21.15.122:0/2451563908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64cc06d180 msgr2=0x7f64b80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:03.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64cbfff640 1 --2- 172.21.15.122:0/2451563908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64cc06d180 0x7f64b80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:03.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64cbfff640 1 -- 172.21.15.122:0/2451563908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64b8003980 msgr2=0x7f64b8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:03.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64cbfff640 1 --2- 172.21.15.122:0/2451563908 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64b8003980 0x7f64b8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:03.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64cbfff640 1 -- 172.21.15.122:0/2451563908 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b8003530 con 0x7f64b8004d10 2024-03-16T20:45:03.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64caffd640 1 --2- 172.21.15.122:0/2451563908 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64cc06d180 0x7f64b80047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:03.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64cbfff640 1 --2- 172.21.15.122:0/2451563908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64b8004d10 0x7f64b8005190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f64bc007390 tx=0x7f64bc01ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8214bcb436cdbfb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:03.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64ca7fc640 1 -- 172.21.15.122:0/2451563908 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc0a5d40 con 0x7f64b8004d10 2024-03-16T20:45:03.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64ca7fc640 1 -- 172.21.15.122:0/2451563908 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64bc0ad440 con 0x7f64b8004d10 2024-03-16T20:45:03.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64ca7fc640 1 -- 172.21.15.122:0/2451563908 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc0a5ea0 con 0x7f64b8004d10 2024-03-16T20:45:03.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64d1cf8640 1 -- 172.21.15.122:0/2451563908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64b8004d10 msgr2=0x7f64b8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:03.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64d1cf8640 1 --2- 172.21.15.122:0/2451563908 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64b8004d10 0x7f64b8005190 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f64bc007390 tx=0x7f64bc01ce40 comp rx=0 tx=0).stop 2024-03-16T20:45:03.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64cbfff640 1 -- 172.21.15.122:0/2451563908 reap_dead start 2024-03-16T20:45:03.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64d1cf8640 1 -- 172.21.15.122:0/2451563908 shutdown_connections 2024-03-16T20:45:03.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64d1cf8640 1 -- 172.21.15.122:0/2451563908 >> 172.21.15.122:0/2451563908 conn(0x7f64cc0bca20 msgr2=0x7f64cc0bce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:03.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64d1cf8640 1 -- 172.21.15.122:0/2451563908 shutdown_connections 2024-03-16T20:45:03.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.497+0000 7f64d1cf8640 1 -- 172.21.15.122:0/2451563908 wait complete. 2024-03-16T20:45:03.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64d1cf8640 1 Processor -- start 2024-03-16T20:45:03.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64d1cf8640 1 -- start start 2024-03-16T20:45:03.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64d1cf8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64cc09b420 0x7f64cc0b44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:03.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64d1cf8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64cc0b49e0 0x7f64cc1df9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:03.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64d1cf8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64cc0b6180 0x7f64cc1dff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:03.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64d1cf8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc09b800 con 0x7f64cc0b6180 2024-03-16T20:45:03.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64d1cf8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc0b5cf0 con 0x7f64cc0b49e0 2024-03-16T20:45:03.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64d1cf8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc0b6730 con 0x7f64cc09b420 2024-03-16T20:45:03.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64cbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64cc0b6180 0x7f64cc1dff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:03.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64cbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64cc0b6180 0x7f64cc1dff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33340/0 (socket says 172.21.15.122:33340) 2024-03-16T20:45:03.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64cbfff640 1 -- 172.21.15.122:0/685560169 learned_addr learned my addr 172.21.15.122:0/685560169 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:03.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64caffd640 1 --2- 172.21.15.122:0/685560169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64cc0b49e0 0x7f64cc1df9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:03.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64cb7fe640 1 --2- 172.21.15.122:0/685560169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64cc09b420 0x7f64cc0b44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:03.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64cbfff640 1 -- 172.21.15.122:0/685560169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64cc09b420 msgr2=0x7f64cc0b44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:03.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64cbfff640 1 --2- 172.21.15.122:0/685560169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64cc09b420 0x7f64cc0b44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:03.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64cbfff640 1 -- 172.21.15.122:0/685560169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64cc0b49e0 msgr2=0x7f64cc1df9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:03.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64cbfff640 1 --2- 172.21.15.122:0/685560169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64cc0b49e0 0x7f64cc1df9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:03.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64cbfff640 1 -- 172.21.15.122:0/685560169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc0b6a10 con 0x7f64cc0b6180 2024-03-16T20:45:03.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64cbfff640 1 --2- 172.21.15.122:0/685560169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64cc0b6180 0x7f64cc1dff00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f64bc002240 tx=0x7f64b8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:03.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64c3fff640 1 -- 172.21.15.122:0/685560169 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc0ad9f0 con 0x7f64cc0b6180 2024-03-16T20:45:03.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64c3fff640 1 -- 172.21.15.122:0/685560169 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64bc0adb50 con 0x7f64cc0b6180 2024-03-16T20:45:03.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64d1cf8640 1 -- 172.21.15.122:0/685560169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64cc0b5a10 con 0x7f64cc0b6180 2024-03-16T20:45:03.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64c3fff640 1 -- 172.21.15.122:0/685560169 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc0b6430 con 0x7f64cc0b6180 2024-03-16T20:45:03.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64d1cf8640 1 -- 172.21.15.122:0/685560169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc09bd50 con 0x7f64cc0b6180 2024-03-16T20:45:03.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64d1cf8640 1 -- 172.21.15.122:0/685560169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64cc074bb0 con 0x7f64cc0b6180 2024-03-16T20:45:03.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64c3fff640 1 -- 172.21.15.122:0/685560169 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f64bc0b4040 con 0x7f64cc0b6180 2024-03-16T20:45:03.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64c3fff640 1 --2- 172.21.15.122:0/685560169 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f64ac066e40 0x7f64ac069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:03.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.501+0000 7f64c3fff640 1 -- 172.21.15.122:0/685560169 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f64bc11ee40 con 0x7f64cc0b6180 2024-03-16T20:45:03.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.505+0000 7f64cb7fe640 1 --2- 172.21.15.122:0/685560169 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f64ac066e40 0x7f64ac069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:03.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.505+0000 7f64cb7fe640 1 --2- 172.21.15.122:0/685560169 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f64ac066e40 0x7f64ac069300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f64c4004850 tx=0x7f64c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:03.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.509+0000 7f64c3fff640 1 -- 172.21.15.122:0/685560169 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f64bc0ebfc0 con 0x7f64cc0b6180 2024-03-16T20:45:03.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:03.669+0000 7f64d1cf8640 1 -- 172.21.15.122:0/685560169 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f64cc06be20 con 0x7f64cc0b6180 2024-03-16T20:45:04.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.337+0000 7f64c3fff640 1 -- 172.21.15.122:0/685560169 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f64bc0ef680 con 0x7f64cc0b6180 2024-03-16T20:45:04.345 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' created 2024-03-16T20:45:04.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.345+0000 7f64d1cf8640 1 -- 172.21.15.122:0/685560169 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f64ac066e40 msgr2=0x7f64ac069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:04.349 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.345+0000 7f64d1cf8640 1 --2- 172.21.15.122:0/685560169 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f64ac066e40 0x7f64ac069300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f64c4004850 tx=0x7f64c4009040 comp rx=0 tx=0).stop 2024-03-16T20:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.345+0000 7f64d1cf8640 1 -- 172.21.15.122:0/685560169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64cc0b6180 msgr2=0x7f64cc1dff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.345+0000 7f64d1cf8640 1 --2- 172.21.15.122:0/685560169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64cc0b6180 0x7f64cc1dff00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f64bc002240 tx=0x7f64b8005800 comp rx=0 tx=0).stop 2024-03-16T20:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.345+0000 7f64d1cf8640 1 -- 172.21.15.122:0/685560169 shutdown_connections 2024-03-16T20:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.345+0000 7f64d1cf8640 1 --2- 172.21.15.122:0/685560169 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f64ac066e40 0x7f64ac069300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.345+0000 7f64d1cf8640 1 --2- 172.21.15.122:0/685560169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64cc0b6180 0x7f64cc1dff00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.345+0000 7f64d1cf8640 1 --2- 172.21.15.122:0/685560169 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64cc0b49e0 0x7f64cc1df9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.345+0000 7f64d1cf8640 1 --2- 172.21.15.122:0/685560169 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f64cc09b420 0x7f64cc0b44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.345+0000 7f64d1cf8640 1 -- 172.21.15.122:0/685560169 >> 172.21.15.122:0/685560169 conn(0x7f64cc0bca20 msgr2=0x7f64cc13b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.345+0000 7f64d1cf8640 1 -- 172.21.15.122:0/685560169 shutdown_connections 2024-03-16T20:45:04.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.345+0000 7f64d1cf8640 1 -- 172.21.15.122:0/685560169 wait complete. 2024-03-16T20:45:04.386 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-16T20:45:04.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.497+0000 7fc3e0763640 1 Processor -- start 2024-03-16T20:45:04.504 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.497+0000 7fc3e0763640 1 -- start start 2024-03-16T20:45:04.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.497+0000 7fc3e0763640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3d809ae30 0x7fc3d80b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:04.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.497+0000 7fc3e0763640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc3d8144aa0 0x7fc3d814e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:04.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.497+0000 7fc3e0763640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d809b210 con 0x7fc3d80aef80 2024-03-16T20:45:04.505 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.497+0000 7fc3e0763640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d80b3110 con 0x7fc3d8144aa0 2024-03-16T20:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.497+0000 7fc3e0763640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80b3970 con 0x7fc3d809ae30 2024-03-16T20:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.497+0000 7fc3ded00640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc3d8144aa0 0x7fc3d814e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.497+0000 7fc3ded00640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc3d8144aa0 0x7fc3d814e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33352/0 (socket says 172.21.15.122:33352) 2024-03-16T20:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.497+0000 7fc3ded00640 1 -- 172.21.15.122:0/2769344163 learned_addr learned my addr 172.21.15.122:0/2769344163 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.497+0000 7fc3de4ff640 1 --2- 172.21.15.122:0/2769344163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3d809ae30 0x7fc3d80b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.497+0000 7fc3ded00640 1 -- 172.21.15.122:0/2769344163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3d809ae30 msgr2=0x7fc3d80b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.497+0000 7fc3ded00640 1 --2- 172.21.15.122:0/2769344163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3d809ae30 0x7fc3d80b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.497+0000 7fc3ded00640 1 -- 172.21.15.122:0/2769344163 >> v1:172.21.15.195:6789/0 conn(0x7fc3d80aef80 legacy=0x7fc3d8144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.497+0000 7fc3ded00640 1 -- 172.21.15.122:0/2769344163 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d80b3320 con 0x7fc3d8144aa0 2024-03-16T20:45:04.506 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.497+0000 7fc3de4ff640 1 --2- 172.21.15.122:0/2769344163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3d809ae30 0x7fc3d80b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:04.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.501+0000 7fc3ded00640 1 --2- 172.21.15.122:0/2769344163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3d8144aa0 0x7fc3d814e850 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009e50 tx=0x7fc3d001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b51b236d3d15f3a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:04.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.501+0000 7fc3dd4fd640 1 -- 172.21.15.122:0/2769344163 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d00a56f0 con 0x7fc3d8144aa0 2024-03-16T20:45:04.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.501+0000 7fc3dd4fd640 1 -- 172.21.15.122:0/2769344163 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d00a5850 con 0x7fc3d8144aa0 2024-03-16T20:45:04.507 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.501+0000 7fc3dd4fd640 1 -- 172.21.15.122:0/2769344163 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d00a5be0 con 0x7fc3d8144aa0 2024-03-16T20:45:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.501+0000 7fc3e0763640 1 -- 172.21.15.122:0/2769344163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3d8144aa0 msgr2=0x7fc3d814e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.501+0000 7fc3e0763640 1 --2- 172.21.15.122:0/2769344163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3d8144aa0 0x7fc3d814e850 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009e50 tx=0x7fc3d001c960 comp rx=0 tx=0).stop 2024-03-16T20:45:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.501+0000 7fc3e0763640 1 -- 172.21.15.122:0/2769344163 shutdown_connections 2024-03-16T20:45:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.501+0000 7fc3e0763640 1 --2- 172.21.15.122:0/2769344163 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3d8144aa0 0x7fc3d814e850 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:04.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.501+0000 7fc3e0763640 1 --2- 172.21.15.122:0/2769344163 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3d809ae30 0x7fc3d80b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:04.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.501+0000 7fc3e0763640 1 -- 172.21.15.122:0/2769344163 >> 172.21.15.122:0/2769344163 conn(0x7fc3d80b8d10 msgr2=0x7fc3d809a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:04.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.501+0000 7fc3e0763640 1 -- 172.21.15.122:0/2769344163 shutdown_connections 2024-03-16T20:45:04.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.501+0000 7fc3e0763640 1 -- 172.21.15.122:0/2769344163 wait complete. 2024-03-16T20:45:04.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3e0763640 1 Processor -- start 2024-03-16T20:45:04.509 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3e0763640 1 -- start start 2024-03-16T20:45:04.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3e0763640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3d809ae30 0x7fc3d813e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:04.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3e0763640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3d80aef80 0x7fc3d80b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:04.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3e0763640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3d8144aa0 0x7fc3d80b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:04.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3e0763640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80afeb0 con 0x7fc3d80aef80 2024-03-16T20:45:04.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3e0763640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d80afbb0 con 0x7fc3d809ae30 2024-03-16T20:45:04.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3e0763640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d81404c0 con 0x7fc3d8144aa0 2024-03-16T20:45:04.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3ddcfe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3d80aef80 0x7fc3d80b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:04.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3ddcfe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3d80aef80 0x7fc3d80b1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33368/0 (socket says 172.21.15.122:33368) 2024-03-16T20:45:04.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3ddcfe640 1 -- 172.21.15.122:0/935562840 learned_addr learned my addr 172.21.15.122:0/935562840 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:04.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3de4ff640 1 --2- 172.21.15.122:0/935562840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3d809ae30 0x7fc3d813e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:04.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3ded00640 1 --2- 172.21.15.122:0/935562840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3d8144aa0 0x7fc3d80b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:04.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3ddcfe640 1 -- 172.21.15.122:0/935562840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3d8144aa0 msgr2=0x7fc3d80b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:04.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3ddcfe640 1 --2- 172.21.15.122:0/935562840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3d8144aa0 0x7fc3d80b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:04.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3ddcfe640 1 -- 172.21.15.122:0/935562840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3d809ae30 msgr2=0x7fc3d813e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:04.512 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3ddcfe640 1 --2- 172.21.15.122:0/935562840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3d809ae30 0x7fc3d813e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:04.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3ddcfe640 1 -- 172.21.15.122:0/935562840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d0009b30 con 0x7fc3d80aef80 2024-03-16T20:45:04.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3ddcfe640 1 --2- 172.21.15.122:0/935562840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3d80aef80 0x7fc3d80b1760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800c450 tx=0x7fc3c800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:04.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3ceffd640 1 -- 172.21.15.122:0/935562840 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8012830 con 0x7fc3d80aef80 2024-03-16T20:45:04.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3ceffd640 1 -- 172.21.15.122:0/935562840 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c80129b0 con 0x7fc3d80aef80 2024-03-16T20:45:04.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3ceffd640 1 -- 172.21.15.122:0/935562840 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8012d60 con 0x7fc3d80aef80 2024-03-16T20:45:04.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3e0763640 1 -- 172.21.15.122:0/935562840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d80b0bf0 con 0x7fc3d80aef80 2024-03-16T20:45:04.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3e0763640 1 -- 172.21.15.122:0/935562840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d809a210 con 0x7fc3d80aef80 2024-03-16T20:45:04.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3e0763640 1 -- 172.21.15.122:0/935562840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d8074bb0 con 0x7fc3d80aef80 2024-03-16T20:45:04.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3ceffd640 1 -- 172.21.15.122:0/935562840 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fc3c801c460 con 0x7fc3d80aef80 2024-03-16T20:45:04.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3ceffd640 1 --2- 172.21.15.122:0/935562840 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc3bc066e40 0x7fc3bc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:04.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3ceffd640 1 -- 172.21.15.122:0/935562840 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fc3c80847a0 con 0x7fc3d80aef80 2024-03-16T20:45:04.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.505+0000 7fc3de4ff640 1 --2- 172.21.15.122:0/935562840 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc3bc066e40 0x7fc3bc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:04.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.509+0000 7fc3de4ff640 1 --2- 172.21.15.122:0/935562840 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc3bc066e40 0x7fc3bc069300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc3d81445c0 tx=0x7fc3d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:04.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.509+0000 7fc3ceffd640 1 -- 172.21.15.122:0/935562840 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc3c8051680 con 0x7fc3d80aef80 2024-03-16T20:45:04.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:04.705+0000 7fc3e0763640 1 -- 172.21.15.122:0/935562840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc3d806e2c0 con 0x7fc3d80aef80 2024-03-16T20:45:05.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.401+0000 7fc3ceffd640 1 -- 172.21.15.122:0/935562840 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc3c8054d40 con 0x7fc3d80aef80 2024-03-16T20:45:05.407 INFO:teuthology.orchestra.run.smithi122.stderr:new fs with metadata pool 2 and data pool 3 2024-03-16T20:45:05.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.409+0000 7fc3e0763640 1 -- 172.21.15.122:0/935562840 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc3bc066e40 msgr2=0x7fc3bc069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:05.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.409+0000 7fc3e0763640 1 --2- 172.21.15.122:0/935562840 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc3bc066e40 0x7fc3bc069300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc3d81445c0 tx=0x7fc3d4008040 comp rx=0 tx=0).stop 2024-03-16T20:45:05.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.409+0000 7fc3e0763640 1 -- 172.21.15.122:0/935562840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3d80aef80 msgr2=0x7fc3d80b1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:05.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.409+0000 7fc3e0763640 1 --2- 172.21.15.122:0/935562840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3d80aef80 0x7fc3d80b1760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc3c800c450 tx=0x7fc3c800c910 comp rx=0 tx=0).stop 2024-03-16T20:45:05.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.413+0000 7fc3e0763640 1 -- 172.21.15.122:0/935562840 shutdown_connections 2024-03-16T20:45:05.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.413+0000 7fc3e0763640 1 --2- 172.21.15.122:0/935562840 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc3bc066e40 0x7fc3bc069300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.413+0000 7fc3e0763640 1 --2- 172.21.15.122:0/935562840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc3d8144aa0 0x7fc3d80b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.413+0000 7fc3e0763640 1 --2- 172.21.15.122:0/935562840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc3d80aef80 0x7fc3d80b1760 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.413+0000 7fc3e0763640 1 --2- 172.21.15.122:0/935562840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc3d809ae30 0x7fc3d813e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.418 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.413+0000 7fc3e0763640 1 -- 172.21.15.122:0/935562840 >> 172.21.15.122:0/935562840 conn(0x7fc3d80b8d10 msgr2=0x7fc3d80b7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:05.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.417+0000 7fc3e0763640 1 -- 172.21.15.122:0/935562840 shutdown_connections 2024-03-16T20:45:05.422 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.417+0000 7fc3e0763640 1 -- 172.21.15.122:0/935562840 wait complete. 2024-03-16T20:45:05.451 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:05.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.561+0000 7f78d0f14640 1 Processor -- start 2024-03-16T20:45:05.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.561+0000 7f78d0f14640 1 -- start start 2024-03-16T20:45:05.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.561+0000 7f78d0f14640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f78cc0b2690 0x7f78cc0b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:05.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.561+0000 7f78d0f14640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78cc143dd0 0x7f78cc147be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:05.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.561+0000 7f78d0f14640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f78cc071a20 0x7f78cc1445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:05.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.561+0000 7f78d0f14640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78cc13e5e0 con 0x7f78cc0b2690 2024-03-16T20:45:05.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.561+0000 7f78d0f14640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78cc0b46e0 con 0x7f78cc071a20 2024-03-16T20:45:05.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.561+0000 7f78d0f14640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc0b2a70 con 0x7f78cc143dd0 2024-03-16T20:45:05.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78ca59c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f78cc0b2690 0x7f78cc0b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:05.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78ca59c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f78cc0b2690 0x7f78cc0b41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33370/0 (socket says 172.21.15.122:33370) 2024-03-16T20:45:05.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78ca59c640 1 -- 172.21.15.122:0/3850953904 learned_addr learned my addr 172.21.15.122:0/3850953904 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:05.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78cad9d640 1 --2- 172.21.15.122:0/3850953904 >> v2:172.21.15.195:3300/0 conn(0x7f78cc071a20 0x7f78cc1445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:05.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78c9d9b640 1 --2- 172.21.15.122:0/3850953904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78cc143dd0 0x7f78cc147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:05.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78ca59c640 1 -- 172.21.15.122:0/3850953904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78cc143dd0 msgr2=0x7f78cc147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:05.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78ca59c640 1 --2- 172.21.15.122:0/3850953904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78cc143dd0 0x7f78cc147be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78ca59c640 1 -- 172.21.15.122:0/3850953904 >> v2:172.21.15.195:3300/0 conn(0x7f78cc071a20 msgr2=0x7f78cc1445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:05.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78ca59c640 1 --2- 172.21.15.122:0/3850953904 >> v2:172.21.15.195:3300/0 conn(0x7f78cc071a20 0x7f78cc1445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78ca59c640 1 -- 172.21.15.122:0/3850953904 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78cc0b4950 con 0x7f78cc0b2690 2024-03-16T20:45:05.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78cad9d640 1 --2- 172.21.15.122:0/3850953904 >> v2:172.21.15.195:3300/0 conn(0x7f78cc071a20 0x7f78cc1445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:05.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78ca59c640 1 --2- 172.21.15.122:0/3850953904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78cc0b2690 0x7f78cc0b41a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0b5140 tx=0x7f78bc01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393200371e913483 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:05.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78c959a640 1 -- 172.21.15.122:0/3850953904 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc0aa610 con 0x7f78cc0b2690 2024-03-16T20:45:05.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78c959a640 1 -- 172.21.15.122:0/3850953904 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78bc0a6040 con 0x7f78cc0b2690 2024-03-16T20:45:05.572 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78c959a640 1 -- 172.21.15.122:0/3850953904 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc0aa990 con 0x7f78cc0b2690 2024-03-16T20:45:05.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78d0f14640 1 -- 172.21.15.122:0/3850953904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78cc0b2690 msgr2=0x7f78cc0b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:05.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78d0f14640 1 --2- 172.21.15.122:0/3850953904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78cc0b2690 0x7f78cc0b41a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0b5140 tx=0x7f78bc01c4f0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78d0f14640 1 -- 172.21.15.122:0/3850953904 shutdown_connections 2024-03-16T20:45:05.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78d0f14640 1 --2- 172.21.15.122:0/3850953904 >> v2:172.21.15.195:3300/0 conn(0x7f78cc071a20 0x7f78cc1445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78d0f14640 1 --2- 172.21.15.122:0/3850953904 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78cc143dd0 0x7f78cc147be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78d0f14640 1 --2- 172.21.15.122:0/3850953904 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78cc0b2690 0x7f78cc0b41a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78d0f14640 1 -- 172.21.15.122:0/3850953904 >> 172.21.15.122:0/3850953904 conn(0x7f78cc0b8860 msgr2=0x7f78cc0b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:05.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78d0f14640 1 -- 172.21.15.122:0/3850953904 shutdown_connections 2024-03-16T20:45:05.573 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.565+0000 7f78d0f14640 1 -- 172.21.15.122:0/3850953904 wait complete. 2024-03-16T20:45:05.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78d0f14640 1 Processor -- start 2024-03-16T20:45:05.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78d0f14640 1 -- start start 2024-03-16T20:45:05.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78d0f14640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78cc071a20 0x7f78cc1379b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:05.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78d0f14640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78cc0b2690 0x7f78cc1e10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:05.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78d0f14640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78cc143dd0 0x7f78cc146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:05.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78d0f14640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc0b82c0 con 0x7f78cc143dd0 2024-03-16T20:45:05.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78d0f14640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc1e0c40 con 0x7f78cc0b2690 2024-03-16T20:45:05.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78d0f14640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc1e1610 con 0x7f78cc071a20 2024-03-16T20:45:05.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78cad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78cc143dd0 0x7f78cc146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:05.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78cad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78cc143dd0 0x7f78cc146830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33382/0 (socket says 172.21.15.122:33382) 2024-03-16T20:45:05.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78cad9d640 1 -- 172.21.15.122:0/3801375586 learned_addr learned my addr 172.21.15.122:0/3801375586 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:05.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78c9d9b640 1 --2- 172.21.15.122:0/3801375586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78cc0b2690 0x7f78cc1e10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:05.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78ca59c640 1 --2- 172.21.15.122:0/3801375586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78cc071a20 0x7f78cc1379b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:05.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78cad9d640 1 -- 172.21.15.122:0/3801375586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78cc071a20 msgr2=0x7f78cc1379b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:05.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78cad9d640 1 --2- 172.21.15.122:0/3801375586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78cc071a20 0x7f78cc1379b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78cad9d640 1 -- 172.21.15.122:0/3801375586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78cc0b2690 msgr2=0x7f78cc1e10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:05.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78cad9d640 1 --2- 172.21.15.122:0/3801375586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78cc0b2690 0x7f78cc1e10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78cad9d640 1 -- 172.21.15.122:0/3801375586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78bc0094d0 con 0x7f78cc143dd0 2024-03-16T20:45:05.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78cad9d640 1 --2- 172.21.15.122:0/3801375586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78cc143dd0 0x7f78cc146830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f78c400cf20 tx=0x7f78c400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:05.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78baffd640 1 -- 172.21.15.122:0/3801375586 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c400d840 con 0x7f78cc143dd0 2024-03-16T20:45:05.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78baffd640 1 -- 172.21.15.122:0/3801375586 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c400d9a0 con 0x7f78cc143dd0 2024-03-16T20:45:05.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78baffd640 1 -- 172.21.15.122:0/3801375586 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c400dd50 con 0x7f78cc143dd0 2024-03-16T20:45:05.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78d0f14640 1 -- 172.21.15.122:0/3801375586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78cc138110 con 0x7f78cc143dd0 2024-03-16T20:45:05.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78d0f14640 1 -- 172.21.15.122:0/3801375586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78cc146d70 con 0x7f78cc143dd0 2024-03-16T20:45:05.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78d0f14640 1 -- 172.21.15.122:0/3801375586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78cc1e0730 con 0x7f78cc143dd0 2024-03-16T20:45:05.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.569+0000 7f78baffd640 1 -- 172.21.15.122:0/3801375586 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f78c401e3b0 con 0x7f78cc143dd0 2024-03-16T20:45:05.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.573+0000 7f78baffd640 1 --2- 172.21.15.122:0/3801375586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f78ac066e40 0x7f78ac069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:05.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.573+0000 7f78baffd640 1 -- 172.21.15.122:0/3801375586 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78c4085500 con 0x7f78cc143dd0 2024-03-16T20:45:05.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.573+0000 7f78ca59c640 1 --2- 172.21.15.122:0/3801375586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f78ac066e40 0x7f78ac069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:05.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.573+0000 7f78ca59c640 1 --2- 172.21.15.122:0/3801375586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f78ac066e40 0x7f78ac069300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0ae000 tx=0x7f78bc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:05.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.577+0000 7f78baffd640 1 -- 172.21.15.122:0/3801375586 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f78cc1e0730 con 0x7f78cc143dd0 2024-03-16T20:45:05.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.737+0000 7f78d0f14640 1 -- 172.21.15.122:0/3801375586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78cc1e0d80 con 0x7f78cc143dd0 2024-03-16T20:45:05.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.741+0000 7f78baffd640 1 -- 172.21.15.122:0/3801375586 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16654 (secure 0 0 0) 0x7f78c40522e0 con 0x7f78cc143dd0 2024-03-16T20:45:05.745 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:05.746 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":20,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:04.711148+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:05.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.741+0000 7f78d0f14640 1 -- 172.21.15.122:0/3801375586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f78ac066e40 msgr2=0x7f78ac069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:05.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.741+0000 7f78d0f14640 1 --2- 172.21.15.122:0/3801375586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f78ac066e40 0x7f78ac069300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0ae000 tx=0x7f78bc008040 comp rx=0 tx=0).stop 2024-03-16T20:45:05.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.741+0000 7f78d0f14640 1 -- 172.21.15.122:0/3801375586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78cc143dd0 msgr2=0x7f78cc146830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:05.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.741+0000 7f78d0f14640 1 --2- 172.21.15.122:0/3801375586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78cc143dd0 0x7f78cc146830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f78c400cf20 tx=0x7f78c400a7d0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.741+0000 7f78d0f14640 1 -- 172.21.15.122:0/3801375586 shutdown_connections 2024-03-16T20:45:05.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.741+0000 7f78d0f14640 1 --2- 172.21.15.122:0/3801375586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f78ac066e40 0x7f78ac069300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.741+0000 7f78d0f14640 1 --2- 172.21.15.122:0/3801375586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f78cc143dd0 0x7f78cc146830 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.741+0000 7f78d0f14640 1 --2- 172.21.15.122:0/3801375586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f78cc0b2690 0x7f78cc1e10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.741+0000 7f78d0f14640 1 --2- 172.21.15.122:0/3801375586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78cc071a20 0x7f78cc1379b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.741+0000 7f78d0f14640 1 -- 172.21.15.122:0/3801375586 >> 172.21.15.122:0/3801375586 conn(0x7f78cc0b8860 msgr2=0x7f78cc1410a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:05.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.741+0000 7f78d0f14640 1 -- 172.21.15.122:0/3801375586 shutdown_connections 2024-03-16T20:45:05.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.741+0000 7f78d0f14640 1 -- 172.21.15.122:0/3801375586 wait complete. 2024-03-16T20:45:05.764 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:05.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.877+0000 7f690bd77640 1 Processor -- start 2024-03-16T20:45:05.885 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f690bd77640 1 -- start start 2024-03-16T20:45:05.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f690bd77640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f69040b4400 0x7f6904140da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:05.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f690bd77640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690406fd90 con 0x7f69040b4860 2024-03-16T20:45:05.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f690bd77640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69040af860 con 0x7f690406b1f0 2024-03-16T20:45:05.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f690bd77640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690406b5d0 con 0x7f69040b4400 2024-03-16T20:45:05.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f690a314640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f69040b4860 0x7f69041412e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:34052/0 (socket says 172.21.15.122:34052) 2024-03-16T20:45:05.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f690a314640 1 -- 172.21.15.122:0/778437702 learned_addr learned my addr 172.21.15.122:0/778437702 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:05.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6909312640 1 --2- 172.21.15.122:0/778437702 >> v2:172.21.15.195:3300/0 conn(0x7f69040b4400 0x7f6904140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:05.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6908b11640 1 -- 172.21.15.122:0/778437702 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249415956 0 0) 0x7f690406fd90 con 0x7f69040b4860 2024-03-16T20:45:05.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6908b11640 1 -- 172.21.15.122:0/778437702 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68f0003530 con 0x7f69040b4860 2024-03-16T20:45:05.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6908b11640 1 -- 172.21.15.122:0/778437702 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3839345921 0 0) 0x7f68f0003530 con 0x7f69040b4860 2024-03-16T20:45:05.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6908b11640 1 -- 172.21.15.122:0/778437702 >> v2:172.21.15.195:3300/0 conn(0x7f69040b4400 msgr2=0x7f6904140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:05.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6908b11640 1 --2- 172.21.15.122:0/778437702 >> v2:172.21.15.195:3300/0 conn(0x7f69040b4400 0x7f6904140da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6908b11640 1 -- 172.21.15.122:0/778437702 >> v1:172.21.15.195:6789/0 conn(0x7f690406b1f0 legacy=0x7f69040af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:05.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6908b11640 1 -- 172.21.15.122:0/778437702 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6904070e10 con 0x7f69040b4860 2024-03-16T20:45:05.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6908b11640 1 -- 172.21.15.122:0/778437702 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f68f4001680 con 0x7f69040b4860 2024-03-16T20:45:05.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6908b11640 1 -- 172.21.15.122:0/778437702 >> v1:172.21.15.122:6789/0 conn(0x7f69040b4860 legacy=0x7f69041412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:05.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6908b11640 1 --2- 172.21.15.122:0/778437702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68f00039d0 0x7f68f0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:05.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6908b11640 1 --2- 172.21.15.122:0/778437702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68f00047d0 0x7f68f0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:05.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6908b11640 1 --2- 172.21.15.122:0/778437702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68f00052b0 0x7f68f000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:05.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6908b11640 1 -- 172.21.15.122:0/778437702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690406b5d0 con 0x7f68f00039d0 2024-03-16T20:45:05.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6908b11640 1 -- 172.21.15.122:0/778437702 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f000c1b0 con 0x7f68f00047d0 2024-03-16T20:45:05.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6908b11640 1 -- 172.21.15.122:0/778437702 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f000c370 con 0x7f68f00052b0 2024-03-16T20:45:05.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6909b13640 1 --2- 172.21.15.122:0/778437702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68f00039d0 0x7f68f0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:05.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f690a314640 1 --2- 172.21.15.122:0/778437702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68f00052b0 0x7f68f000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:05.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6909b13640 1 -- 172.21.15.122:0/778437702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68f00052b0 msgr2=0x7f68f000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:05.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6909312640 1 --2- 172.21.15.122:0/778437702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68f00047d0 0x7f68f0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:05.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6909b13640 1 --2- 172.21.15.122:0/778437702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68f00052b0 0x7f68f000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6909b13640 1 -- 172.21.15.122:0/778437702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68f00047d0 msgr2=0x7f68f0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:05.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6909b13640 1 --2- 172.21.15.122:0/778437702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68f00047d0 0x7f68f0004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f6909b13640 1 -- 172.21.15.122:0/778437702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f0003530 con 0x7f68f00039d0 2024-03-16T20:45:05.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f690a314640 1 --2- 172.21.15.122:0/778437702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68f00052b0 0x7f68f000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:05.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.881+0000 7f690a314640 1 -- 172.21.15.122:0/778437702 reap_dead start 2024-03-16T20:45:05.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f6909b13640 1 --2- 172.21.15.122:0/778437702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68f00039d0 0x7f68f0003df0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f68fc001780 tx=0x7f68fc01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0adb0e593bde8f8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:05.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f6908b11640 1 -- 172.21.15.122:0/778437702 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc0052d0 con 0x7f68f00039d0 2024-03-16T20:45:05.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f6908b11640 1 -- 172.21.15.122:0/778437702 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68fc0a7050 con 0x7f68f00039d0 2024-03-16T20:45:05.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f6908b11640 1 -- 172.21.15.122:0/778437702 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc003600 con 0x7f68f00039d0 2024-03-16T20:45:05.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690bd77640 1 -- 172.21.15.122:0/778437702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68f00039d0 msgr2=0x7f68f0003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:05.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690bd77640 1 --2- 172.21.15.122:0/778437702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68f00039d0 0x7f68f0003df0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f68fc001780 tx=0x7f68fc01c3d0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690bd77640 1 -- 172.21.15.122:0/778437702 shutdown_connections 2024-03-16T20:45:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690bd77640 1 --2- 172.21.15.122:0/778437702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68f00039d0 0x7f68f0003df0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690bd77640 1 -- 172.21.15.122:0/778437702 >> 172.21.15.122:0/778437702 conn(0x7f69040b8860 msgr2=0x7f69040b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690bd77640 1 -- 172.21.15.122:0/778437702 shutdown_connections 2024-03-16T20:45:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690bd77640 1 -- 172.21.15.122:0/778437702 wait complete. 2024-03-16T20:45:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690bd77640 1 Processor -- start 2024-03-16T20:45:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690bd77640 1 -- start start 2024-03-16T20:45:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690bd77640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68f00039d0 0x7f6904149880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690bd77640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69041df430 0x7f6904149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690bd77640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69040b1e90 0x7f69040b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690bd77640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690414ae40 con 0x7f68f00039d0 2024-03-16T20:45:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690bd77640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6904137900 con 0x7f69041df430 2024-03-16T20:45:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690bd77640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69041e0390 con 0x7f69040b1e90 2024-03-16T20:45:05.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f6909b13640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68f00039d0 0x7f6904149880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f6909b13640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68f00039d0 0x7f6904149880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33412/0 (socket says 172.21.15.122:33412) 2024-03-16T20:45:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f6909b13640 1 -- 172.21.15.122:0/203665907 learned_addr learned my addr 172.21.15.122:0/203665907 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f6909312640 1 --2- 172.21.15.122:0/203665907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69041df430 0x7f6904149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690a314640 1 --2- 172.21.15.122:0/203665907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69040b1e90 0x7f69040b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f6909b13640 1 -- 172.21.15.122:0/203665907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69040b1e90 msgr2=0x7f69040b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f6909b13640 1 --2- 172.21.15.122:0/203665907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69040b1e90 0x7f69040b2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f6909b13640 1 -- 172.21.15.122:0/203665907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69041df430 msgr2=0x7f6904149dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f6909b13640 1 --2- 172.21.15.122:0/203665907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69041df430 0x7f6904149dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f6909b13640 1 -- 172.21.15.122:0/203665907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69041df8b0 con 0x7f68f00039d0 2024-03-16T20:45:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f6909b13640 1 --2- 172.21.15.122:0/203665907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68f00039d0 0x7f6904149880 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f68fc01c200 tx=0x7f68f0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f68fa7fc640 1 -- 172.21.15.122:0/203665907 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc0072f0 con 0x7f68f00039d0 2024-03-16T20:45:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f68fa7fc640 1 -- 172.21.15.122:0/203665907 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68fc007470 con 0x7f68f00039d0 2024-03-16T20:45:05.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690bd77640 1 -- 172.21.15.122:0/203665907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69041dfb90 con 0x7f68f00039d0 2024-03-16T20:45:05.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f68fa7fc640 1 -- 172.21.15.122:0/203665907 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc0067e0 con 0x7f68f00039d0 2024-03-16T20:45:05.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.885+0000 7f690bd77640 1 -- 172.21.15.122:0/203665907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690414b040 con 0x7f68f00039d0 2024-03-16T20:45:05.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.889+0000 7f690bd77640 1 -- 172.21.15.122:0/203665907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6904075280 con 0x7f68f00039d0 2024-03-16T20:45:05.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.889+0000 7f68fa7fc640 1 -- 172.21.15.122:0/203665907 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f68fc006940 con 0x7f68f00039d0 2024-03-16T20:45:05.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.889+0000 7f68fa7fc640 1 --2- 172.21.15.122:0/203665907 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f68e8066e40 0x7f68e8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:05.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.889+0000 7f68fa7fc640 1 -- 172.21.15.122:0/203665907 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68fc11ea60 con 0x7f68f00039d0 2024-03-16T20:45:05.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.889+0000 7f6909312640 1 --2- 172.21.15.122:0/203665907 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f68e8066e40 0x7f68e8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:05.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.889+0000 7f6909312640 1 --2- 172.21.15.122:0/203665907 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f68e8066e40 0x7f68e8069300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6900009fd0 tx=0x7f690000b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:05.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:05.893+0000 7f68fa7fc640 1 -- 172.21.15.122:0/203665907 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f68fc0eb850 con 0x7f68f00039d0 2024-03-16T20:45:06.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.049+0000 7f690bd77640 1 -- 172.21.15.122:0/203665907 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69040ae210 con 0x7f68f00039d0 2024-03-16T20:45:06.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.053+0000 7f68fa7fc640 1 -- 172.21.15.122:0/203665907 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16654 (secure 0 0 0) 0x7f68fc0eef10 con 0x7f68f00039d0 2024-03-16T20:45:06.058 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:06.059 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":20,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:04.711148+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.053+0000 7f690bd77640 1 -- 172.21.15.122:0/203665907 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f68e8066e40 msgr2=0x7f68e8069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.053+0000 7f690bd77640 1 --2- 172.21.15.122:0/203665907 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f68e8066e40 0x7f68e8069300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6900009fd0 tx=0x7f690000b040 comp rx=0 tx=0).stop 2024-03-16T20:45:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.053+0000 7f690bd77640 1 -- 172.21.15.122:0/203665907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68f00039d0 msgr2=0x7f6904149880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.053+0000 7f690bd77640 1 --2- 172.21.15.122:0/203665907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68f00039d0 0x7f6904149880 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f68fc01c200 tx=0x7f68f0005800 comp rx=0 tx=0).stop 2024-03-16T20:45:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.053+0000 7f690bd77640 1 -- 172.21.15.122:0/203665907 shutdown_connections 2024-03-16T20:45:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.053+0000 7f690bd77640 1 --2- 172.21.15.122:0/203665907 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f68e8066e40 0x7f68e8069300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.053+0000 7f690bd77640 1 --2- 172.21.15.122:0/203665907 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f69040b1e90 0x7f69040b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.053+0000 7f690bd77640 1 --2- 172.21.15.122:0/203665907 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f69041df430 0x7f6904149dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.053+0000 7f690bd77640 1 --2- 172.21.15.122:0/203665907 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68f00039d0 0x7f6904149880 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.053+0000 7f690bd77640 1 -- 172.21.15.122:0/203665907 >> 172.21.15.122:0/203665907 conn(0x7f69040b8860 msgr2=0x7f690413ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:06.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.053+0000 7f690bd77640 1 -- 172.21.15.122:0/203665907 shutdown_connections 2024-03-16T20:45:06.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.053+0000 7f690bd77640 1 -- 172.21.15.122:0/203665907 wait complete. 2024-03-16T20:45:06.076 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-16T20:45:06.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.181+0000 7f0f57102640 1 Processor -- start 2024-03-16T20:45:06.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.181+0000 7f0f57102640 1 -- start start 2024-03-16T20:45:06.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.181+0000 7f0f57102640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0f5009ae30 0x7f0f500b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:06.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f57102640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0f500b7ba0 0x7f0f500b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:06.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f57102640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f500b3fa0 con 0x7f0f501422e0 2024-03-16T20:45:06.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f57102640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f500b50a0 con 0x7f0f5009ae30 2024-03-16T20:45:06.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f57102640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f500b8590 con 0x7f0f500b7ba0 2024-03-16T20:45:06.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f54e9e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0f5009ae30 0x7f0f500b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:06.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f54e9e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0f5009ae30 0x7f0f500b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33414/0 (socket says 172.21.15.122:33414) 2024-03-16T20:45:06.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f54e9e640 1 -- 172.21.15.122:0/1695987620 learned_addr learned my addr 172.21.15.122:0/1695987620 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:06.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f54e9e640 1 -- 172.21.15.122:0/1695987620 >> v2:172.21.15.195:3300/0 conn(0x7f0f500b7ba0 msgr2=0x7f0f500b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:06.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f54e9e640 1 --2- 172.21.15.122:0/1695987620 >> v2:172.21.15.195:3300/0 conn(0x7f0f500b7ba0 0x7f0f500b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f54e9e640 1 -- 172.21.15.122:0/1695987620 >> v1:172.21.15.195:6789/0 conn(0x7f0f501422e0 legacy=0x7f0f50142770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:06.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f54e9e640 1 -- 172.21.15.122:0/1695987620 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5009b210 con 0x7f0f5009ae30 2024-03-16T20:45:06.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f54e9e640 1 --2- 172.21.15.122:0/1695987620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5009ae30 0x7f0f500b4430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f5006e120 tx=0x7f0f4801c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbba3b01500a44cf server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:06.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f4f7fe640 1 -- 172.21.15.122:0/1695987620 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480a5870 con 0x7f0f5009ae30 2024-03-16T20:45:06.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f4f7fe640 1 -- 172.21.15.122:0/1695987620 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f480a59d0 con 0x7f0f5009ae30 2024-03-16T20:45:06.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f4f7fe640 1 -- 172.21.15.122:0/1695987620 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480a5d60 con 0x7f0f5009ae30 2024-03-16T20:45:06.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f57102640 1 -- 172.21.15.122:0/1695987620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5009ae30 msgr2=0x7f0f500b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:06.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f57102640 1 --2- 172.21.15.122:0/1695987620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5009ae30 0x7f0f500b4430 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0f5006e120 tx=0x7f0f4801c9b0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f57102640 1 -- 172.21.15.122:0/1695987620 shutdown_connections 2024-03-16T20:45:06.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f57102640 1 --2- 172.21.15.122:0/1695987620 >> v2:172.21.15.195:3300/0 conn(0x7f0f500b7ba0 0x7f0f500b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f57102640 1 --2- 172.21.15.122:0/1695987620 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5009ae30 0x7f0f500b4430 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f57102640 1 -- 172.21.15.122:0/1695987620 >> 172.21.15.122:0/1695987620 conn(0x7f0f500b8d10 msgr2=0x7f0f5009a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:06.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f57102640 1 -- 172.21.15.122:0/1695987620 shutdown_connections 2024-03-16T20:45:06.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.185+0000 7f0f57102640 1 -- 172.21.15.122:0/1695987620 wait complete. 2024-03-16T20:45:06.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f57102640 1 Processor -- start 2024-03-16T20:45:06.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f57102640 1 -- start start 2024-03-16T20:45:06.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f57102640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5009ae30 0x7f0f5009c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:06.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f57102640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f500b7ba0 0x7f0f5013eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:06.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f57102640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f501422e0 0x7f0f501e0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:06.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f57102640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5013cae0 con 0x7f0f5009ae30 2024-03-16T20:45:06.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f57102640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5013d2f0 con 0x7f0f501422e0 2024-03-16T20:45:06.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f57102640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5013f420 con 0x7f0f500b7ba0 2024-03-16T20:45:06.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f54e9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5009ae30 0x7f0f5009c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:06.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f54e9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5009ae30 0x7f0f5009c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33422/0 (socket says 172.21.15.122:33422) 2024-03-16T20:45:06.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f54e9e640 1 -- 172.21.15.122:0/1154652433 learned_addr learned my addr 172.21.15.122:0/1154652433 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:06.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f54e9e640 1 -- 172.21.15.122:0/1154652433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f500b7ba0 msgr2=0x7f0f5013eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:06.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f54e9e640 1 --2- 172.21.15.122:0/1154652433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f500b7ba0 0x7f0f5013eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f54e9e640 1 -- 172.21.15.122:0/1154652433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f501422e0 msgr2=0x7f0f501e0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:06.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f5569f640 1 --2- 172.21.15.122:0/1154652433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f501422e0 0x7f0f501e0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:06.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f54e9e640 1 --2- 172.21.15.122:0/1154652433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f501422e0 0x7f0f501e0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f54e9e640 1 -- 172.21.15.122:0/1154652433 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f48009b30 con 0x7f0f5009ae30 2024-03-16T20:45:06.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f54e9e640 1 --2- 172.21.15.122:0/1154652433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5009ae30 0x7f0f5009c300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f4801b060 tx=0x7f0f48019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:06.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f4d7fa640 1 -- 172.21.15.122:0/1154652433 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480b8880 con 0x7f0f5009ae30 2024-03-16T20:45:06.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f4d7fa640 1 -- 172.21.15.122:0/1154652433 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f480b89e0 con 0x7f0f5009ae30 2024-03-16T20:45:06.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f57102640 1 -- 172.21.15.122:0/1154652433 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5013f6a0 con 0x7f0f5009ae30 2024-03-16T20:45:06.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f4d7fa640 1 -- 172.21.15.122:0/1154652433 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480b8d90 con 0x7f0f5009ae30 2024-03-16T20:45:06.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f57102640 1 -- 172.21.15.122:0/1154652433 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5013d010 con 0x7f0f5009ae30 2024-03-16T20:45:06.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f57102640 1 -- 172.21.15.122:0/1154652433 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f501e07b0 con 0x7f0f5009ae30 2024-03-16T20:45:06.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f4d7fa640 1 -- 172.21.15.122:0/1154652433 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f0f480c1410 con 0x7f0f5009ae30 2024-03-16T20:45:06.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f4d7fa640 1 --2- 172.21.15.122:0/1154652433 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0f30066e70 0x7f0f30069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:06.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f4d7fa640 1 -- 172.21.15.122:0/1154652433 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f4811fde0 con 0x7f0f5009ae30 2024-03-16T20:45:06.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.189+0000 7f0f4ffff640 1 --2- 172.21.15.122:0/1154652433 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0f30066e70 0x7f0f30069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:06.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.193+0000 7f0f4ffff640 1 --2- 172.21.15.122:0/1154652433 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0f30066e70 0x7f0f30069330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0f50072e80 tx=0x7f0f38006ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:06.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.193+0000 7f0f4d7fa640 1 -- 172.21.15.122:0/1154652433 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0f501e07b0 con 0x7f0f5009ae30 2024-03-16T20:45:06.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.373+0000 7f0f57102640 1 -- 172.21.15.122:0/1154652433 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0f501e07b0 con 0x7f0f5009ae30 2024-03-16T20:45:06.417 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.409+0000 7f0f4d7fa640 1 -- 172.21.15.122:0/1154652433 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0f501e07b0 con 0x7f0f5009ae30 2024-03-16T20:45:06.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.413+0000 7f0f57102640 1 -- 172.21.15.122:0/1154652433 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0f30066e70 msgr2=0x7f0f30069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:06.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.413+0000 7f0f57102640 1 --2- 172.21.15.122:0/1154652433 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0f30066e70 0x7f0f30069330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0f50072e80 tx=0x7f0f38006ca0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.413+0000 7f0f57102640 1 -- 172.21.15.122:0/1154652433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5009ae30 msgr2=0x7f0f5009c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:06.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.413+0000 7f0f57102640 1 --2- 172.21.15.122:0/1154652433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5009ae30 0x7f0f5009c300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0f4801b060 tx=0x7f0f48019750 comp rx=0 tx=0).stop 2024-03-16T20:45:06.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.413+0000 7f0f57102640 1 -- 172.21.15.122:0/1154652433 shutdown_connections 2024-03-16T20:45:06.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.413+0000 7f0f57102640 1 --2- 172.21.15.122:0/1154652433 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0f30066e70 0x7f0f30069330 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.413+0000 7f0f57102640 1 --2- 172.21.15.122:0/1154652433 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0f501422e0 0x7f0f501e0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.413+0000 7f0f57102640 1 --2- 172.21.15.122:0/1154652433 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0f500b7ba0 0x7f0f5013eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.413+0000 7f0f57102640 1 --2- 172.21.15.122:0/1154652433 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0f5009ae30 0x7f0f5009c300 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.413+0000 7f0f57102640 1 -- 172.21.15.122:0/1154652433 >> 172.21.15.122:0/1154652433 conn(0x7f0f500b8d10 msgr2=0x7f0f5006d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:06.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.413+0000 7f0f57102640 1 -- 172.21.15.122:0/1154652433 shutdown_connections 2024-03-16T20:45:06.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.413+0000 7f0f57102640 1 -- 172.21.15.122:0/1154652433 wait complete. 2024-03-16T20:45:06.436 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-16T20:45:06.436 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-16T20:45:06.464 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T20:45:06.482 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T20:45:06.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f3016467640 1 Processor -- start 2024-03-16T20:45:06.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f3016467640 1 -- start start 2024-03-16T20:45:06.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f3016467640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f3010141580 0x7f30100baac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:06.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f3016467640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f30100bb000 0x7f301006cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:06.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f3016467640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f301006d180 0x7f301014a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:06.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f3016467640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3010147ff0 con 0x7f3010141580 2024-03-16T20:45:06.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f3016467640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30100bb460 con 0x7f30100bb000 2024-03-16T20:45:06.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f3016467640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30101479a0 con 0x7f301006d180 2024-03-16T20:45:06.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f300ffff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f3010141580 0x7f30100baac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:06.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f300ffff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f3010141580 0x7f30100baac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33432/0 (socket says 172.21.15.122:33432) 2024-03-16T20:45:06.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f300ffff640 1 -- 172.21.15.122:0/1902664952 learned_addr learned my addr 172.21.15.122:0/1902664952 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:06.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f3014a04640 1 --2- 172.21.15.122:0/1902664952 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f301006d180 0x7f301014a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:06.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f30077fe640 1 --2- 172.21.15.122:0/1902664952 >> v2:172.21.15.195:3300/0 conn(0x7f30100bb000 0x7f301006cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:06.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f300ffff640 1 -- 172.21.15.122:0/1902664952 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f301006d180 msgr2=0x7f301014a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:06.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f300ffff640 1 --2- 172.21.15.122:0/1902664952 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f301006d180 0x7f301014a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f300ffff640 1 -- 172.21.15.122:0/1902664952 >> v2:172.21.15.195:3300/0 conn(0x7f30100bb000 msgr2=0x7f301006cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:06.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f300ffff640 1 --2- 172.21.15.122:0/1902664952 >> v2:172.21.15.195:3300/0 conn(0x7f30100bb000 0x7f301006cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f300ffff640 1 -- 172.21.15.122:0/1902664952 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3010147c80 con 0x7f3010141580 2024-03-16T20:45:06.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.549+0000 7f30077fe640 1 --2- 172.21.15.122:0/1902664952 >> v2:172.21.15.195:3300/0 conn(0x7f30100bb000 0x7f301006cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:06.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.553+0000 7f300ffff640 1 --2- 172.21.15.122:0/1902664952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3010141580 0x7f30100baac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f301006c380 tx=0x7f300001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16082f998eb9a71f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:06.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.553+0000 7f300f7fe640 1 -- 172.21.15.122:0/1902664952 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30000ab600 con 0x7f3010141580 2024-03-16T20:45:06.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.553+0000 7f300f7fe640 1 -- 172.21.15.122:0/1902664952 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30000a7050 con 0x7f3010141580 2024-03-16T20:45:06.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.553+0000 7f300f7fe640 1 -- 172.21.15.122:0/1902664952 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30000ab980 con 0x7f3010141580 2024-03-16T20:45:06.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.553+0000 7f3016467640 1 -- 172.21.15.122:0/1902664952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3010141580 msgr2=0x7f30100baac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:06.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.553+0000 7f3016467640 1 --2- 172.21.15.122:0/1902664952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3010141580 0x7f30100baac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f301006c380 tx=0x7f300001c520 comp rx=0 tx=0).stop 2024-03-16T20:45:06.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.553+0000 7f3016467640 1 -- 172.21.15.122:0/1902664952 shutdown_connections 2024-03-16T20:45:06.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.553+0000 7f3016467640 1 --2- 172.21.15.122:0/1902664952 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f301006d180 0x7f301014a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.553+0000 7f3016467640 1 --2- 172.21.15.122:0/1902664952 >> v2:172.21.15.195:3300/0 conn(0x7f30100bb000 0x7f301006cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.553+0000 7f3016467640 1 --2- 172.21.15.122:0/1902664952 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3010141580 0x7f30100baac0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.553+0000 7f3016467640 1 -- 172.21.15.122:0/1902664952 >> 172.21.15.122:0/1902664952 conn(0x7f30100bca20 msgr2=0x7f30100bce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:06.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.553+0000 7f3016467640 1 -- 172.21.15.122:0/1902664952 shutdown_connections 2024-03-16T20:45:06.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.553+0000 7f3016467640 1 -- 172.21.15.122:0/1902664952 wait complete. 2024-03-16T20:45:06.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.553+0000 7f3016467640 1 Processor -- start 2024-03-16T20:45:06.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.553+0000 7f3016467640 1 -- start start 2024-03-16T20:45:06.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f3016467640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f301006d180 0x7f30100b5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:06.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f3016467640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30100bb000 0x7f301013ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:06.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f3016467640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3010141580 0x7f301013b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:06.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f3016467640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301013b690 con 0x7f30100bb000 2024-03-16T20:45:06.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f3016467640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30100b6c80 con 0x7f3010141580 2024-03-16T20:45:06.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f3016467640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30100b3d20 con 0x7f301006d180 2024-03-16T20:45:06.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f30077fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30100bb000 0x7f301013ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:06.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f30077fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30100bb000 0x7f301013ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33446/0 (socket says 172.21.15.122:33446) 2024-03-16T20:45:06.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f30077fe640 1 -- 172.21.15.122:0/3283726296 learned_addr learned my addr 172.21.15.122:0/3283726296 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f300ffff640 1 --2- 172.21.15.122:0/3283726296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f301006d180 0x7f30100b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f3014a04640 1 --2- 172.21.15.122:0/3283726296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3010141580 0x7f301013b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f30077fe640 1 -- 172.21.15.122:0/3283726296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f301006d180 msgr2=0x7f30100b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f30077fe640 1 --2- 172.21.15.122:0/3283726296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f301006d180 0x7f30100b5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f30077fe640 1 -- 172.21.15.122:0/3283726296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3010141580 msgr2=0x7f301013b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f30077fe640 1 --2- 172.21.15.122:0/3283726296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3010141580 0x7f301013b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f30077fe640 1 -- 172.21.15.122:0/3283726296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30000094d0 con 0x7f30100bb000 2024-03-16T20:45:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f300ffff640 1 --2- 172.21.15.122:0/3283726296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f301006d180 0x7f30100b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f30077fe640 1 --2- 172.21.15.122:0/3283726296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30100bb000 0x7f301013ac10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800c7f0 tx=0x7f2ff800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f300d7fa640 1 -- 172.21.15.122:0/3283726296 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff800cea0 con 0x7f30100bb000 2024-03-16T20:45:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f300d7fa640 1 -- 172.21.15.122:0/3283726296 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ff8007a30 con 0x7f30100bb000 2024-03-16T20:45:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f3016467640 1 -- 172.21.15.122:0/3283726296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30100b4e00 con 0x7f30100bb000 2024-03-16T20:45:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f300d7fa640 1 -- 172.21.15.122:0/3283726296 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8007de0 con 0x7f30100bb000 2024-03-16T20:45:06.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f3016467640 1 -- 172.21.15.122:0/3283726296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30100b3f70 con 0x7f30100bb000 2024-03-16T20:45:06.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f3016467640 1 -- 172.21.15.122:0/3283726296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30100b69a0 con 0x7f30100bb000 2024-03-16T20:45:06.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f300d7fa640 1 -- 172.21.15.122:0/3283726296 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f2ff80113f0 con 0x7f30100bb000 2024-03-16T20:45:06.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f300d7fa640 1 --2- 172.21.15.122:0/3283726296 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2ff0066e40 0x7f2ff0069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:06.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f300d7fa640 1 -- 172.21.15.122:0/3283726296 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ff8084a60 con 0x7f30100bb000 2024-03-16T20:45:06.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f300ffff640 1 --2- 172.21.15.122:0/3283726296 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2ff0066e40 0x7f2ff0069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:06.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.557+0000 7f300ffff640 1 --2- 172.21.15.122:0/3283726296 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2ff0066e40 0x7f2ff0069300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f30000ad000 tx=0x7f3000008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:06.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.561+0000 7f300d7fa640 1 -- 172.21.15.122:0/3283726296 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f30100b69a0 con 0x7f30100bb000 2024-03-16T20:45:06.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:06.761+0000 7f3016467640 1 -- 172.21.15.122:0/3283726296 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f30100b69a0 con 0x7f30100bb000 2024-03-16T20:45:07.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.421+0000 7f300d7fa640 1 -- 172.21.15.122:0/3283726296 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f30100b69a0 con 0x7f30100bb000 2024-03-16T20:45:07.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.425+0000 7f3016467640 1 -- 172.21.15.122:0/3283726296 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2ff0066e40 msgr2=0x7f2ff0069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:07.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.425+0000 7f3016467640 1 --2- 172.21.15.122:0/3283726296 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2ff0066e40 0x7f2ff0069300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f30000ad000 tx=0x7f3000008040 comp rx=0 tx=0).stop 2024-03-16T20:45:07.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.425+0000 7f3016467640 1 -- 172.21.15.122:0/3283726296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30100bb000 msgr2=0x7f301013ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:07.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.425+0000 7f3016467640 1 --2- 172.21.15.122:0/3283726296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30100bb000 0x7f301013ac10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800c7f0 tx=0x7f2ff800ccb0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.425+0000 7f3016467640 1 -- 172.21.15.122:0/3283726296 shutdown_connections 2024-03-16T20:45:07.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.425+0000 7f3016467640 1 --2- 172.21.15.122:0/3283726296 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2ff0066e40 0x7f2ff0069300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.425+0000 7f3016467640 1 --2- 172.21.15.122:0/3283726296 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3010141580 0x7f301013b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.425+0000 7f3016467640 1 --2- 172.21.15.122:0/3283726296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30100bb000 0x7f301013ac10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.425+0000 7f3016467640 1 --2- 172.21.15.122:0/3283726296 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f301006d180 0x7f30100b5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.425+0000 7f3016467640 1 -- 172.21.15.122:0/3283726296 >> 172.21.15.122:0/3283726296 conn(0x7f30100bca20 msgr2=0x7f301013ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:07.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.425+0000 7f3016467640 1 -- 172.21.15.122:0/3283726296 shutdown_connections 2024-03-16T20:45:07.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.425+0000 7f3016467640 1 -- 172.21.15.122:0/3283726296 wait complete. 2024-03-16T20:45:07.448 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-16T20:45:07.449 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T20:45:07.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.605+0000 7f7050049640 1 Processor -- start 2024-03-16T20:45:07.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.605+0000 7f7050049640 1 -- start start 2024-03-16T20:45:07.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f7050049640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70480b2690 0x7f70480b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f7050049640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7048143dd0 0x7f7048147be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f7050049640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70480b46e0 con 0x7f7048071a20 2024-03-16T20:45:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f7050049640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f704813e5e0 con 0x7f7048143dd0 2024-03-16T20:45:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f7050049640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70480b2a70 con 0x7f70480b2690 2024-03-16T20:45:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f704d5e4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7048143dd0 0x7f7048147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f704e5e6640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f7048071a20 0x7f70481445a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:34058/0 (socket says 172.21.15.122:34058) 2024-03-16T20:45:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f704e5e6640 1 -- 172.21.15.122:0/4140486018 learned_addr learned my addr 172.21.15.122:0/4140486018 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f704dde5640 1 --2- 172.21.15.122:0/4140486018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70480b2690 0x7f70480b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f704d5e4640 1 -- 172.21.15.122:0/4140486018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70480b2690 msgr2=0x7f70480b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f704d5e4640 1 --2- 172.21.15.122:0/4140486018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70480b2690 0x7f70480b41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f704d5e4640 1 -- 172.21.15.122:0/4140486018 >> v1:172.21.15.122:6789/0 conn(0x7f7048071a20 legacy=0x7f70481445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:07.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f704d5e4640 1 -- 172.21.15.122:0/4140486018 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70480b4950 con 0x7f7048143dd0 2024-03-16T20:45:07.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f704d5e4640 1 --2- 172.21.15.122:0/4140486018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7048143dd0 0x7f7048147be0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f70380097c0 tx=0x7f703801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6639c5e91d246ae server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:07.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f704cde3640 1 -- 172.21.15.122:0/4140486018 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70380ab600 con 0x7f7048143dd0 2024-03-16T20:45:07.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f704cde3640 1 -- 172.21.15.122:0/4140486018 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70380a7050 con 0x7f7048143dd0 2024-03-16T20:45:07.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f704cde3640 1 -- 172.21.15.122:0/4140486018 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70380ab980 con 0x7f7048143dd0 2024-03-16T20:45:07.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f7050049640 1 -- 172.21.15.122:0/4140486018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7048143dd0 msgr2=0x7f7048147be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:07.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f7050049640 1 --2- 172.21.15.122:0/4140486018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7048143dd0 0x7f7048147be0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f70380097c0 tx=0x7f703801c330 comp rx=0 tx=0).stop 2024-03-16T20:45:07.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f7050049640 1 -- 172.21.15.122:0/4140486018 shutdown_connections 2024-03-16T20:45:07.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f7050049640 1 --2- 172.21.15.122:0/4140486018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7048143dd0 0x7f7048147be0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f7050049640 1 --2- 172.21.15.122:0/4140486018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f70480b2690 0x7f70480b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f7050049640 1 -- 172.21.15.122:0/4140486018 >> 172.21.15.122:0/4140486018 conn(0x7f70480b8860 msgr2=0x7f70480b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:07.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.609+0000 7f7050049640 1 -- 172.21.15.122:0/4140486018 shutdown_connections 2024-03-16T20:45:07.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f7050049640 1 -- 172.21.15.122:0/4140486018 wait complete. 2024-03-16T20:45:07.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f7050049640 1 Processor -- start 2024-03-16T20:45:07.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f7050049640 1 -- start start 2024-03-16T20:45:07.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f7050049640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7048071a20 0x7f704813d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:07.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f7050049640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70480b2690 0x7f70481510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:07.620 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f7050049640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7048143dd0 0x7f70481316d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:07.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f7050049640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70481dfd10 con 0x7f7048143dd0 2024-03-16T20:45:07.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f7050049640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7048145180 con 0x7f70480b2690 2024-03-16T20:45:07.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f7050049640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7048144f30 con 0x7f7048071a20 2024-03-16T20:45:07.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f704e5e6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7048143dd0 0x7f70481316d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:07.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f704e5e6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7048143dd0 0x7f70481316d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33454/0 (socket says 172.21.15.122:33454) 2024-03-16T20:45:07.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f704e5e6640 1 -- 172.21.15.122:0/3206479883 learned_addr learned my addr 172.21.15.122:0/3206479883 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:07.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f704d5e4640 1 --2- 172.21.15.122:0/3206479883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70480b2690 0x7f70481510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:07.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f704dde5640 1 --2- 172.21.15.122:0/3206479883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7048071a20 0x7f704813d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:07.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f704e5e6640 1 -- 172.21.15.122:0/3206479883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7048071a20 msgr2=0x7f704813d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:07.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f704e5e6640 1 --2- 172.21.15.122:0/3206479883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7048071a20 0x7f704813d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f704e5e6640 1 -- 172.21.15.122:0/3206479883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70480b2690 msgr2=0x7f70481510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:07.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f704e5e6640 1 --2- 172.21.15.122:0/3206479883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70480b2690 0x7f70481510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f704e5e6640 1 -- 172.21.15.122:0/3206479883 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7038009420 con 0x7f7048143dd0 2024-03-16T20:45:07.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f704dde5640 1 --2- 172.21.15.122:0/3206479883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7048071a20 0x7f704813d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:07.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f704e5e6640 1 --2- 172.21.15.122:0/3206479883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7048143dd0 0x7f70481316d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f704000e7b0 tx=0x7f704000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:07.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f703e7fc640 1 -- 172.21.15.122:0/3206479883 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704000f930 con 0x7f7048143dd0 2024-03-16T20:45:07.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f703e7fc640 1 -- 172.21.15.122:0/3206479883 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704000fa90 con 0x7f7048143dd0 2024-03-16T20:45:07.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f703e7fc640 1 -- 172.21.15.122:0/3206479883 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704000fe00 con 0x7f7048143dd0 2024-03-16T20:45:07.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f7050049640 1 -- 172.21.15.122:0/3206479883 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7048146e20 con 0x7f7048143dd0 2024-03-16T20:45:07.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f7050049640 1 -- 172.21.15.122:0/3206479883 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f704813dd30 con 0x7f7048143dd0 2024-03-16T20:45:07.622 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f7050049640 1 -- 172.21.15.122:0/3206479883 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7048146c80 con 0x7f7048143dd0 2024-03-16T20:45:07.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.613+0000 7f703e7fc640 1 -- 172.21.15.122:0/3206479883 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f7040015070 con 0x7f7048143dd0 2024-03-16T20:45:07.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.617+0000 7f703e7fc640 1 --2- 172.21.15.122:0/3206479883 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f702c066e40 0x7f702c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:07.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.617+0000 7f703e7fc640 1 -- 172.21.15.122:0/3206479883 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f70400873f0 con 0x7f7048143dd0 2024-03-16T20:45:07.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.617+0000 7f704dde5640 1 --2- 172.21.15.122:0/3206479883 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f702c066e40 0x7f702c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:07.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.617+0000 7f704dde5640 1 --2- 172.21.15.122:0/3206479883 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f702c066e40 0x7f702c069300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7048135050 tx=0x7f7044008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:07.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.621+0000 7f703e7fc640 1 -- 172.21.15.122:0/3206479883 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7048146c80 con 0x7f7048143dd0 2024-03-16T20:45:07.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.801+0000 7f7050049640 1 -- 172.21.15.122:0/3206479883 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f704814f710 con 0x7f7048143dd0 2024-03-16T20:45:07.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.801+0000 7f703e7fc640 1 -- 172.21.15.122:0/3206479883 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3397 (secure 0 0 0) 0x7f704814f710 con 0x7f7048143dd0 2024-03-16T20:45:07.807 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:07.807 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3481092793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3481092793},{"type":"v1","addr":"172.21.15.122:6833","nonce":3481092793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T20:45:04.711887+0000","modified":"2024-03-16T20:45:06.765685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/1428608180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1428608180},{"type":"v1","addr":"172.21.15.195:6835","nonce":1428608180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-03-16T20:45:07.808 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 6 2024-03-16T20:45:07.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.805+0000 7f7050049640 1 -- 172.21.15.122:0/3206479883 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f702c066e40 msgr2=0x7f702c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:07.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.805+0000 7f7050049640 1 --2- 172.21.15.122:0/3206479883 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f702c066e40 0x7f702c069300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7048135050 tx=0x7f7044008040 comp rx=0 tx=0).stop 2024-03-16T20:45:07.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.805+0000 7f7050049640 1 -- 172.21.15.122:0/3206479883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7048143dd0 msgr2=0x7f70481316d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:07.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.805+0000 7f7050049640 1 --2- 172.21.15.122:0/3206479883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7048143dd0 0x7f70481316d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f704000e7b0 tx=0x7f704000ec70 comp rx=0 tx=0).stop 2024-03-16T20:45:07.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.805+0000 7f7050049640 1 -- 172.21.15.122:0/3206479883 shutdown_connections 2024-03-16T20:45:07.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.805+0000 7f7050049640 1 --2- 172.21.15.122:0/3206479883 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f702c066e40 0x7f702c069300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.805+0000 7f7050049640 1 --2- 172.21.15.122:0/3206479883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7048143dd0 0x7f70481316d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.805+0000 7f7050049640 1 --2- 172.21.15.122:0/3206479883 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f70480b2690 0x7f70481510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.805+0000 7f7050049640 1 --2- 172.21.15.122:0/3206479883 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7048071a20 0x7f704813d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.805+0000 7f7050049640 1 -- 172.21.15.122:0/3206479883 >> 172.21.15.122:0/3206479883 conn(0x7f70480b8860 msgr2=0x7f70480b3520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:07.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.805+0000 7f7050049640 1 -- 172.21.15.122:0/3206479883 shutdown_connections 2024-03-16T20:45:07.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.805+0000 7f7050049640 1 -- 172.21.15.122:0/3206479883 wait complete. 2024-03-16T20:45:07.825 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:07.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.929+0000 7effee4d7640 1 Processor -- start 2024-03-16T20:45:07.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.929+0000 7effee4d7640 1 -- start start 2024-03-16T20:45:07.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.929+0000 7effee4d7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effe8147e90 0x7effe81471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:07.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.929+0000 7effee4d7640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7effe80ba850 0x7effe809b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:07.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.929+0000 7effee4d7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe809aee0 con 0x7effe809ba00 2024-03-16T20:45:07.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.929+0000 7effee4d7640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe80bb1d0 con 0x7effe80ba850 2024-03-16T20:45:07.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.929+0000 7effee4d7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe80b9280 con 0x7effe8147e90 2024-03-16T20:45:07.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.929+0000 7effe77fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7effe80ba850 0x7effe809b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:07.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.929+0000 7effe77fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7effe80ba850 0x7effe809b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33460/0 (socket says 172.21.15.122:33460) 2024-03-16T20:45:07.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.929+0000 7effe77fe640 1 -- 172.21.15.122:0/4126163725 learned_addr learned my addr 172.21.15.122:0/4126163725 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:07.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.929+0000 7effe7fff640 1 --2- 172.21.15.122:0/4126163725 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effe8147e90 0x7effe81471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:07.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.929+0000 7effe77fe640 1 -- 172.21.15.122:0/4126163725 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effe8147e90 msgr2=0x7effe81471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:07.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.929+0000 7effe77fe640 1 --2- 172.21.15.122:0/4126163725 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effe8147e90 0x7effe81471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.929+0000 7effe77fe640 1 -- 172.21.15.122:0/4126163725 >> v1:172.21.15.195:6789/0 conn(0x7effe809ba00 legacy=0x7effe80b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:07.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.929+0000 7effe77fe640 1 -- 172.21.15.122:0/4126163725 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe809ab70 con 0x7effe80ba850 2024-03-16T20:45:07.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.933+0000 7effe77fe640 1 --2- 172.21.15.122:0/4126163725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe80ba850 0x7effe809b370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7effd8009870 tx=0x7effd801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75701b9bd08a8fcd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:07.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.933+0000 7effe6ffd640 1 -- 172.21.15.122:0/4126163725 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8004230 con 0x7effe80ba850 2024-03-16T20:45:07.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.933+0000 7effe6ffd640 1 -- 172.21.15.122:0/4126163725 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effd80a7050 con 0x7effe80ba850 2024-03-16T20:45:07.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.933+0000 7effe6ffd640 1 -- 172.21.15.122:0/4126163725 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effd8003620 con 0x7effe80ba850 2024-03-16T20:45:07.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.933+0000 7effee4d7640 1 -- 172.21.15.122:0/4126163725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe80ba850 msgr2=0x7effe809b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:07.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.933+0000 7effee4d7640 1 --2- 172.21.15.122:0/4126163725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe80ba850 0x7effe809b370 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7effd8009870 tx=0x7effd801c440 comp rx=0 tx=0).stop 2024-03-16T20:45:07.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.933+0000 7effee4d7640 1 -- 172.21.15.122:0/4126163725 shutdown_connections 2024-03-16T20:45:07.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.933+0000 7effee4d7640 1 --2- 172.21.15.122:0/4126163725 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe80ba850 0x7effe809b370 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.933+0000 7effee4d7640 1 --2- 172.21.15.122:0/4126163725 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effe8147e90 0x7effe81471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.933+0000 7effee4d7640 1 -- 172.21.15.122:0/4126163725 >> 172.21.15.122:0/4126163725 conn(0x7effe80bca50 msgr2=0x7effe80bce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:07.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.933+0000 7effee4d7640 1 -- 172.21.15.122:0/4126163725 shutdown_connections 2024-03-16T20:45:07.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.933+0000 7effee4d7640 1 -- 172.21.15.122:0/4126163725 wait complete. 2024-03-16T20:45:07.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.933+0000 7effee4d7640 1 Processor -- start 2024-03-16T20:45:07.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.933+0000 7effee4d7640 1 -- start start 2024-03-16T20:45:07.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effee4d7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe809ba00 0x7effe813fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:07.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effee4d7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effe8147e90 0x7effe809d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:07.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effee4d7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effe813ef20 0x7effe8148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:07.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effee4d7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe81494c0 con 0x7effe809ba00 2024-03-16T20:45:07.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effee4d7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe813f400 con 0x7effe8147e90 2024-03-16T20:45:07.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effee4d7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe81484e0 con 0x7effe813ef20 2024-03-16T20:45:07.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effe7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe809ba00 0x7effe813fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:07.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effe7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe809ba00 0x7effe813fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33474/0 (socket says 172.21.15.122:33474) 2024-03-16T20:45:07.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effe7fff640 1 -- 172.21.15.122:0/3417220973 learned_addr learned my addr 172.21.15.122:0/3417220973 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:07.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effe77fe640 1 --2- 172.21.15.122:0/3417220973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effe8147e90 0x7effe809d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:07.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effeca74640 1 --2- 172.21.15.122:0/3417220973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effe813ef20 0x7effe8148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:07.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effe7fff640 1 -- 172.21.15.122:0/3417220973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effe813ef20 msgr2=0x7effe8148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:07.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effe7fff640 1 --2- 172.21.15.122:0/3417220973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effe813ef20 0x7effe8148f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effe7fff640 1 -- 172.21.15.122:0/3417220973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effe8147e90 msgr2=0x7effe809d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:07.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effe7fff640 1 --2- 172.21.15.122:0/3417220973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effe8147e90 0x7effe809d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:07.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effe7fff640 1 -- 172.21.15.122:0/3417220973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd80094d0 con 0x7effe809ba00 2024-03-16T20:45:07.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effe7fff640 1 --2- 172.21.15.122:0/3417220973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe809ba00 0x7effe813fb70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7effdc00d620 tx=0x7effdc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:07.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effe4ff9640 1 -- 172.21.15.122:0/3417220973 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc00b620 con 0x7effe809ba00 2024-03-16T20:45:07.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effe4ff9640 1 -- 172.21.15.122:0/3417220973 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effdc00b780 con 0x7effe809ba00 2024-03-16T20:45:07.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effee4d7640 1 -- 172.21.15.122:0/3417220973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe809e090 con 0x7effe809ba00 2024-03-16T20:45:07.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effe4ff9640 1 -- 172.21.15.122:0/3417220973 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc00bb10 con 0x7effe809ba00 2024-03-16T20:45:07.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effee4d7640 1 -- 172.21.15.122:0/3417220973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe8148a60 con 0x7effe809ba00 2024-03-16T20:45:07.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effee4d7640 1 -- 172.21.15.122:0/3417220973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe8148340 con 0x7effe809ba00 2024-03-16T20:45:07.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effe4ff9640 1 -- 172.21.15.122:0/3417220973 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7effdc00bc70 con 0x7effe809ba00 2024-03-16T20:45:07.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effe4ff9640 1 --2- 172.21.15.122:0/3417220973 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7effd4066e40 0x7effd4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:07.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effe4ff9640 1 -- 172.21.15.122:0/3417220973 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effdc084e50 con 0x7effe809ba00 2024-03-16T20:45:07.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.937+0000 7effe77fe640 1 --2- 172.21.15.122:0/3417220973 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7effd4066e40 0x7effd4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:07.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.941+0000 7effe77fe640 1 --2- 172.21.15.122:0/3417220973 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7effd4066e40 0x7effd4069300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7effd80ad000 tx=0x7effd8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:07.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:07.941+0000 7effe4ff9640 1 -- 172.21.15.122:0/3417220973 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7effe8148340 con 0x7effe809ba00 2024-03-16T20:45:08.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.097+0000 7effee4d7640 1 -- 172.21.15.122:0/3417220973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7effe8148340 con 0x7effe809ba00 2024-03-16T20:45:08.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.097+0000 7effe4ff9640 1 -- 172.21.15.122:0/3417220973 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7effe8148340 con 0x7effe809ba00 2024-03-16T20:45:08.105 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:08.105 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:06.404479+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:08.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.101+0000 7effee4d7640 1 -- 172.21.15.122:0/3417220973 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7effd4066e40 msgr2=0x7effd4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.101+0000 7effee4d7640 1 --2- 172.21.15.122:0/3417220973 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7effd4066e40 0x7effd4069300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7effd80ad000 tx=0x7effd8009040 comp rx=0 tx=0).stop 2024-03-16T20:45:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.101+0000 7effee4d7640 1 -- 172.21.15.122:0/3417220973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe809ba00 msgr2=0x7effe813fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.101+0000 7effee4d7640 1 --2- 172.21.15.122:0/3417220973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe809ba00 0x7effe813fb70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7effdc00d620 tx=0x7effdc00dae0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.101+0000 7effee4d7640 1 -- 172.21.15.122:0/3417220973 shutdown_connections 2024-03-16T20:45:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.101+0000 7effee4d7640 1 --2- 172.21.15.122:0/3417220973 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7effd4066e40 0x7effd4069300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.101+0000 7effee4d7640 1 --2- 172.21.15.122:0/3417220973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7effe813ef20 0x7effe8148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.101+0000 7effee4d7640 1 --2- 172.21.15.122:0/3417220973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7effe8147e90 0x7effe809d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.101+0000 7effee4d7640 1 --2- 172.21.15.122:0/3417220973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effe809ba00 0x7effe813fb70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.101+0000 7effee4d7640 1 -- 172.21.15.122:0/3417220973 >> 172.21.15.122:0/3417220973 conn(0x7effe80bca50 msgr2=0x7effe8139950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.101+0000 7effee4d7640 1 -- 172.21.15.122:0/3417220973 shutdown_connections 2024-03-16T20:45:08.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.101+0000 7effee4d7640 1 -- 172.21.15.122:0/3417220973 wait complete. 2024-03-16T20:45:08.123 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-16T20:45:08.123 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T20:45:08.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.221+0000 7f23f734e640 1 Processor -- start 2024-03-16T20:45:08.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.221+0000 7f23f734e640 1 -- start start 2024-03-16T20:45:08.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.225+0000 7f23f734e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f23f0072120 0x7f23f00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:08.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.225+0000 7f23f734e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23f00b1050 con 0x7f23f013ed60 2024-03-16T20:45:08.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.225+0000 7f23f734e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23f0072500 con 0x7f23f0148b40 2024-03-16T20:45:08.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.225+0000 7f23f734e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23f00b38a0 con 0x7f23f0072120 2024-03-16T20:45:08.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.225+0000 7f23f50ea640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f23f0072120 0x7f23f00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:08.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.225+0000 7f23f48e9640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f23f013ed60 0x7f23f0148430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:34068/0 (socket says 172.21.15.122:34068) 2024-03-16T20:45:08.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.225+0000 7f23f50ea640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f23f0072120 0x7f23f00afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33478/0 (socket says 172.21.15.122:33478) 2024-03-16T20:45:08.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.225+0000 7f23f48e9640 1 -- 172.21.15.122:0/1919130008 learned_addr learned my addr 172.21.15.122:0/1919130008 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:08.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.225+0000 7f23f50ea640 1 -- 172.21.15.122:0/1919130008 >> v1:172.21.15.195:6789/0 conn(0x7f23f0148b40 legacy=0x7f23f014f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.225+0000 7f23e7fff640 1 -- 172.21.15.122:0/1919130008 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880035942 0 0) 0x7f23f00b1050 con 0x7f23f013ed60 2024-03-16T20:45:08.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.225+0000 7f23f50ea640 1 -- 172.21.15.122:0/1919130008 >> v1:172.21.15.122:6789/0 conn(0x7f23f013ed60 legacy=0x7f23f0148430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.225+0000 7f23f50ea640 1 -- 172.21.15.122:0/1919130008 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f00b3b80 con 0x7f23f0072120 2024-03-16T20:45:08.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.229+0000 7f23f50ea640 1 --2- 172.21.15.122:0/1919130008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f0072120 0x7f23f00afb90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f23e00097c0 tx=0x7f23e001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abbf5b5e9128c47 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:08.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.229+0000 7f23e7fff640 1 -- 172.21.15.122:0/1919130008 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e00ac640 con 0x7f23f0072120 2024-03-16T20:45:08.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.229+0000 7f23e7fff640 1 -- 172.21.15.122:0/1919130008 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23e00ac7a0 con 0x7f23f0072120 2024-03-16T20:45:08.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.229+0000 7f23e7fff640 1 -- 172.21.15.122:0/1919130008 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23e00acb30 con 0x7f23f0072120 2024-03-16T20:45:08.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.229+0000 7f23f734e640 1 -- 172.21.15.122:0/1919130008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f0072120 msgr2=0x7f23f00afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.229+0000 7f23f734e640 1 --2- 172.21.15.122:0/1919130008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f0072120 0x7f23f00afb90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f23e00097c0 tx=0x7f23e001c3e0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.229+0000 7f23f734e640 1 -- 172.21.15.122:0/1919130008 shutdown_connections 2024-03-16T20:45:08.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.229+0000 7f23f734e640 1 --2- 172.21.15.122:0/1919130008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f0072120 0x7f23f00afb90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.229+0000 7f23f734e640 1 -- 172.21.15.122:0/1919130008 >> 172.21.15.122:0/1919130008 conn(0x7f23f00b8860 msgr2=0x7f23f00b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:08.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.229+0000 7f23f734e640 1 -- 172.21.15.122:0/1919130008 shutdown_connections 2024-03-16T20:45:08.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.229+0000 7f23f734e640 1 -- 172.21.15.122:0/1919130008 wait complete. 2024-03-16T20:45:08.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f734e640 1 Processor -- start 2024-03-16T20:45:08.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f734e640 1 -- start start 2024-03-16T20:45:08.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f734e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23f0072120 0x7f23f014fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:08.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f734e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f013ed60 0x7f23f014ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:08.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f734e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23f014f460 0x7f23f014da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:08.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f734e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f01dfe60 con 0x7f23f013ed60 2024-03-16T20:45:08.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f734e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23f0142a00 con 0x7f23f014f460 2024-03-16T20:45:08.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f734e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23f01e03c0 con 0x7f23f0072120 2024-03-16T20:45:08.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f48e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f013ed60 0x7f23f014ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:08.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f48e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f013ed60 0x7f23f014ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33482/0 (socket says 172.21.15.122:33482) 2024-03-16T20:45:08.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f48e9640 1 -- 172.21.15.122:0/1111937082 learned_addr learned my addr 172.21.15.122:0/1111937082 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:08.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f50ea640 1 --2- 172.21.15.122:0/1111937082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23f0072120 0x7f23f014fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:08.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f58eb640 1 --2- 172.21.15.122:0/1111937082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23f014f460 0x7f23f014da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:08.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f48e9640 1 -- 172.21.15.122:0/1111937082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23f0072120 msgr2=0x7f23f014fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f48e9640 1 --2- 172.21.15.122:0/1111937082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23f0072120 0x7f23f014fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f48e9640 1 -- 172.21.15.122:0/1111937082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23f014f460 msgr2=0x7f23f014da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f48e9640 1 --2- 172.21.15.122:0/1111937082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23f014f460 0x7f23f014da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f48e9640 1 -- 172.21.15.122:0/1111937082 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e0009420 con 0x7f23f013ed60 2024-03-16T20:45:08.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f58eb640 1 --2- 172.21.15.122:0/1111937082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23f014f460 0x7f23f014da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:08.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f48e9640 1 --2- 172.21.15.122:0/1111937082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f013ed60 0x7f23f014ef20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00c4a0 tx=0x7f23ec00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:08.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23e5ffb640 1 -- 172.21.15.122:0/1111937082 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec0074e0 con 0x7f23f013ed60 2024-03-16T20:45:08.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23e5ffb640 1 -- 172.21.15.122:0/1111937082 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ec007640 con 0x7f23f013ed60 2024-03-16T20:45:08.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f734e640 1 -- 172.21.15.122:0/1111937082 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23f014f920 con 0x7f23f013ed60 2024-03-16T20:45:08.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23e5ffb640 1 -- 172.21.15.122:0/1111937082 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ec004a30 con 0x7f23f013ed60 2024-03-16T20:45:08.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f734e640 1 -- 172.21.15.122:0/1111937082 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23f00716c0 con 0x7f23f013ed60 2024-03-16T20:45:08.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f734e640 1 -- 172.21.15.122:0/1111937082 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23f01e0500 con 0x7f23f013ed60 2024-03-16T20:45:08.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23e5ffb640 1 -- 172.21.15.122:0/1111937082 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f23ec004b90 con 0x7f23f013ed60 2024-03-16T20:45:08.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23e5ffb640 1 --2- 172.21.15.122:0/1111937082 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f23d4066e40 0x7f23d4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:08.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23e5ffb640 1 -- 172.21.15.122:0/1111937082 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23ec083e90 con 0x7f23f013ed60 2024-03-16T20:45:08.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.233+0000 7f23f50ea640 1 --2- 172.21.15.122:0/1111937082 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f23d4066e40 0x7f23d4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:08.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.237+0000 7f23f50ea640 1 --2- 172.21.15.122:0/1111937082 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f23d4066e40 0x7f23d4069300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f23e00ae000 tx=0x7f23e00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:08.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.237+0000 7f23e5ffb640 1 -- 172.21.15.122:0/1111937082 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f23ec050d10 con 0x7f23f013ed60 2024-03-16T20:45:08.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.380+0000 7f23f734e640 1 -- 172.21.15.122:0/1111937082 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f23f01536c0 con 0x7f23d4066e40 2024-03-16T20:45:08.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.400+0000 7f23e5ffb640 1 -- 172.21.15.122:0/1111937082 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237079 (secure 0 0 0) 0x7f23e00bd090 con 0x7f23d4066e40 2024-03-16T20:45:08.407 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:08.411 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-03-16T20:45:08.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.404+0000 7f23f734e640 1 -- 172.21.15.122:0/1111937082 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f23d4066e40 msgr2=0x7f23d4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.404+0000 7f23f734e640 1 --2- 172.21.15.122:0/1111937082 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f23d4066e40 0x7f23d4069300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f23e00ae000 tx=0x7f23e00a7000 comp rx=0 tx=0).stop 2024-03-16T20:45:08.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.404+0000 7f23f734e640 1 -- 172.21.15.122:0/1111937082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f013ed60 msgr2=0x7f23f014ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.404+0000 7f23f734e640 1 --2- 172.21.15.122:0/1111937082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f013ed60 0x7f23f014ef20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00c4a0 tx=0x7f23ec00c960 comp rx=0 tx=0).stop 2024-03-16T20:45:08.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.404+0000 7f23f734e640 1 -- 172.21.15.122:0/1111937082 shutdown_connections 2024-03-16T20:45:08.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.404+0000 7f23f734e640 1 --2- 172.21.15.122:0/1111937082 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f23d4066e40 0x7f23d4069300 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f23e00ae000 tx=0x7f23e00a7000 comp rx=0 tx=0).stop 2024-03-16T20:45:08.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.404+0000 7f23f734e640 1 --2- 172.21.15.122:0/1111937082 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23f014f460 0x7f23f014da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.404+0000 7f23f734e640 1 --2- 172.21.15.122:0/1111937082 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23f013ed60 0x7f23f014ef20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.404+0000 7f23f734e640 1 --2- 172.21.15.122:0/1111937082 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23f0072120 0x7f23f014fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.404+0000 7f23f734e640 1 -- 172.21.15.122:0/1111937082 >> 172.21.15.122:0/1111937082 conn(0x7f23f00b8860 msgr2=0x7f23f00b5940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:08.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.404+0000 7f23f734e640 1 -- 172.21.15.122:0/1111937082 shutdown_connections 2024-03-16T20:45:08.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.404+0000 7f23f734e640 1 -- 172.21.15.122:0/1111937082 wait complete. 2024-03-16T20:45:08.425 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-03-16T20:45:06.577382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":235620,"kb_used_data":2472,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739100,"statfs":{"total":773094113280,"available":772852838400,"internally_reserved":0,"allocated":2531328,"data_stored":1584240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001072"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-03-16T20:44:55.846201+0000","last_change":"2024-03-16T20:44:54.293418+0000","last_active":"2024-03-16T20:44:55.846201+0000","last_peered":"2024-03-16T20:44:55.846201+0000","last_clean":"2024-03-16T20:44:55.846201+0000","last_became_active":"2024-03-16T20:44:54.291347+0000","last_became_peered":"2024-03-16T20:44:54.291347+0000","last_unstale":"2024-03-16T20:44:55.846201+0000","last_undegraded":"2024-03-16T20:44:55.846201+0000","last_fullsized":"2024-03-16T20:44:55.846201+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_clean_scrub_stamp":"2024-03-16T20:44:53.077639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:29:18.318494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:04.339420+0000","last_change":"2024-03-16T20:45:04.339420+0000","last_active":"2024-03-16T20:45:04.339420+0000","last_peered":"2024-03-16T20:45:04.339420+0000","last_clean":"2024-03-16T20:45:04.339420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:04.339420+0000","last_undegraded":"2024-03-16T20:45:04.339420+0000","last_fullsized":"2024-03-16T20:45:04.339420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:03.331302+0000","last_change":"2024-03-16T20:45:03.331302+0000","last_active":"2024-03-16T20:45:03.331302+0000","last_peered":"2024-03-16T20:45:03.331302+0000","last_clean":"2024-03-16T20:45:03.331302+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:03.331302+0000","last_undegraded":"2024-03-16T20:45:03.331302+0000","last_fullsized":"2024-03-16T20:45:03.331302+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":761856,"data_stored":640806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31348,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340492,"statfs":{"total":96636764160,"available":96604663808,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31352,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31352,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340488,"statfs":{"total":96636764160,"available":96604659712,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":50438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":761856,"data_stored":640806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T20:45:08.429 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T20:45:08.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.568+0000 7fe5d6343640 1 Processor -- start 2024-03-16T20:45:08.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.568+0000 7fe5d6343640 1 -- start start 2024-03-16T20:45:08.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.572+0000 7fe5d6343640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe5d013f240 0x7fe5d0144e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:08.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.572+0000 7fe5d6343640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d0149350 con 0x7fe5d00baca0 2024-03-16T20:45:08.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.572+0000 7fe5d6343640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d00bb080 con 0x7fe5d0148ec0 2024-03-16T20:45:08.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.572+0000 7fe5d6343640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d00b06b0 con 0x7fe5d013f240 2024-03-16T20:45:08.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.572+0000 7fe5cf7fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe5d013f240 0x7fe5d0144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:08.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.572+0000 7fe5cf7fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe5d013f240 0x7fe5d0144e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41144/0 (socket says 172.21.15.122:41144) 2024-03-16T20:45:08.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.572+0000 7fe5cf7fe640 1 -- 172.21.15.122:0/4240128434 learned_addr learned my addr 172.21.15.122:0/4240128434 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:08.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.572+0000 7fe5cf7fe640 1 -- 172.21.15.122:0/4240128434 >> v1:172.21.15.195:6789/0 conn(0x7fe5d0148ec0 legacy=0x7fe5d014f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:08.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.572+0000 7fe5cf7fe640 1 -- 172.21.15.122:0/4240128434 >> v1:172.21.15.122:6789/0 conn(0x7fe5d00baca0 legacy=0x7fe5d013eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:08.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.572+0000 7fe5cf7fe640 1 -- 172.21.15.122:0/4240128434 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d00b1300 con 0x7fe5d013f240 2024-03-16T20:45:08.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.572+0000 7fe5cf7fe640 1 --2- 172.21.15.122:0/4240128434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d013f240 0x7fe5d0144e10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00b1490 tx=0x7fe5c401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1e7e081152499f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:08.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.572+0000 7fe5ceffd640 1 -- 172.21.15.122:0/4240128434 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c40ab650 con 0x7fe5d013f240 2024-03-16T20:45:08.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.572+0000 7fe5ceffd640 1 -- 172.21.15.122:0/4240128434 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c40ab7b0 con 0x7fe5d013f240 2024-03-16T20:45:08.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.572+0000 7fe5ceffd640 1 -- 172.21.15.122:0/4240128434 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c40abb40 con 0x7fe5d013f240 2024-03-16T20:45:08.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 -- 172.21.15.122:0/4240128434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d013f240 msgr2=0x7fe5d0144e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 --2- 172.21.15.122:0/4240128434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d013f240 0x7fe5d0144e10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00b1490 tx=0x7fe5c401c450 comp rx=0 tx=0).stop 2024-03-16T20:45:08.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 -- 172.21.15.122:0/4240128434 shutdown_connections 2024-03-16T20:45:08.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 --2- 172.21.15.122:0/4240128434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d013f240 0x7fe5d0144e10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 -- 172.21.15.122:0/4240128434 >> 172.21.15.122:0/4240128434 conn(0x7fe5d006ee40 msgr2=0x7fe5d006f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:08.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 -- 172.21.15.122:0/4240128434 shutdown_connections 2024-03-16T20:45:08.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 -- 172.21.15.122:0/4240128434 wait complete. 2024-03-16T20:45:08.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 Processor -- start 2024-03-16T20:45:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 -- start start 2024-03-16T20:45:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d00baca0 0x7fe5d014e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5d013f240 0x7fe5d01dfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5d0148ec0 0x7fe5d01e0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d014fd30 con 0x7fe5d00baca0 2024-03-16T20:45:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d00b2800 con 0x7fe5d013f240 2024-03-16T20:45:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d0150010 con 0x7fe5d0148ec0 2024-03-16T20:45:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d00baca0 0x7fe5d014e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d00baca0 0x7fe5d014e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41150/0 (socket says 172.21.15.122:41150) 2024-03-16T20:45:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5cffff640 1 -- 172.21.15.122:0/401175392 learned_addr learned my addr 172.21.15.122:0/401175392 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:08.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5cf7fe640 1 --2- 172.21.15.122:0/401175392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5d013f240 0x7fe5d01dfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d48e0640 1 --2- 172.21.15.122:0/401175392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5d0148ec0 0x7fe5d01e0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5cffff640 1 -- 172.21.15.122:0/401175392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5d0148ec0 msgr2=0x7fe5d01e0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5cffff640 1 --2- 172.21.15.122:0/401175392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5d0148ec0 0x7fe5d01e0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5cffff640 1 -- 172.21.15.122:0/401175392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5d013f240 msgr2=0x7fe5d01dfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5cffff640 1 --2- 172.21.15.122:0/401175392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5d013f240 0x7fe5d01dfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5cffff640 1 -- 172.21.15.122:0/401175392 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c4009480 con 0x7fe5d00baca0 2024-03-16T20:45:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5cffff640 1 --2- 172.21.15.122:0/401175392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d00baca0 0x7fe5d014e3a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000c420 tx=0x7fe5c000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5ccff9640 1 -- 172.21.15.122:0/401175392 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00052d0 con 0x7fe5d00baca0 2024-03-16T20:45:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5ccff9640 1 -- 172.21.15.122:0/401175392 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c0005430 con 0x7fe5d00baca0 2024-03-16T20:45:08.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 -- 172.21.15.122:0/401175392 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d006d7a0 con 0x7fe5d00baca0 2024-03-16T20:45:08.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5ccff9640 1 -- 172.21.15.122:0/401175392 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0007260 con 0x7fe5d00baca0 2024-03-16T20:45:08.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 -- 172.21.15.122:0/401175392 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d006df40 con 0x7fe5d00baca0 2024-03-16T20:45:08.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.576+0000 7fe5d6343640 1 -- 172.21.15.122:0/401175392 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d00677b0 con 0x7fe5d00baca0 2024-03-16T20:45:08.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.580+0000 7fe5ccff9640 1 -- 172.21.15.122:0/401175392 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe5c00073c0 con 0x7fe5d00baca0 2024-03-16T20:45:08.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.580+0000 7fe5ccff9640 1 --2- 172.21.15.122:0/401175392 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe5bc066e40 0x7fe5bc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:08.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.580+0000 7fe5ccff9640 1 -- 172.21.15.122:0/401175392 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5c0083ba0 con 0x7fe5d00baca0 2024-03-16T20:45:08.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.580+0000 7fe5cf7fe640 1 --2- 172.21.15.122:0/401175392 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe5bc066e40 0x7fe5bc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:08.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.580+0000 7fe5cf7fe640 1 --2- 172.21.15.122:0/401175392 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe5bc066e40 0x7fe5bc069300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40af000 tx=0x7fe5c4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:08.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.584+0000 7fe5ccff9640 1 -- 172.21.15.122:0/401175392 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe5c0050a20 con 0x7fe5d00baca0 2024-03-16T20:45:08.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.736+0000 7fe5d6343640 1 -- 172.21.15.122:0/401175392 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5d014fe70 con 0x7fe5bc066e40 2024-03-16T20:45:08.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.752+0000 7fe5ccff9640 1 -- 172.21.15.122:0/401175392 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251919 (secure 0 0 0) 0x7fe5d014fe70 con 0x7fe5bc066e40 2024-03-16T20:45:08.762 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:08.764 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-03-16T20:45:08.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.756+0000 7fe5d6343640 1 -- 172.21.15.122:0/401175392 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe5bc066e40 msgr2=0x7fe5bc069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.756+0000 7fe5d6343640 1 --2- 172.21.15.122:0/401175392 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe5bc066e40 0x7fe5bc069300 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40af000 tx=0x7fe5c4009040 comp rx=0 tx=0).stop 2024-03-16T20:45:08.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.756+0000 7fe5d6343640 1 -- 172.21.15.122:0/401175392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d00baca0 msgr2=0x7fe5d014e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.756+0000 7fe5d6343640 1 --2- 172.21.15.122:0/401175392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d00baca0 0x7fe5d014e3a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe5c000c420 tx=0x7fe5c000c8e0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.756+0000 7fe5d6343640 1 -- 172.21.15.122:0/401175392 shutdown_connections 2024-03-16T20:45:08.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.756+0000 7fe5d6343640 1 --2- 172.21.15.122:0/401175392 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe5bc066e40 0x7fe5bc069300 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.756+0000 7fe5d6343640 1 --2- 172.21.15.122:0/401175392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5d0148ec0 0x7fe5d01e0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.756+0000 7fe5d6343640 1 --2- 172.21.15.122:0/401175392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5d013f240 0x7fe5d01dfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.756+0000 7fe5d6343640 1 --2- 172.21.15.122:0/401175392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe5d00baca0 0x7fe5d014e3a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.756+0000 7fe5d6343640 1 -- 172.21.15.122:0/401175392 >> 172.21.15.122:0/401175392 conn(0x7fe5d006ee40 msgr2=0x7fe5d0143b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:08.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.756+0000 7fe5d6343640 1 -- 172.21.15.122:0/401175392 shutdown_connections 2024-03-16T20:45:08.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.756+0000 7fe5d6343640 1 -- 172.21.15.122:0/401175392 wait complete. 2024-03-16T20:45:08.778 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-03-16T20:45:08.577768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235640,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739080,"statfs":{"total":773094113280,"available":772852817920,"internally_reserved":0,"allocated":2891776,"data_stored":1739996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.237166"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459997+0000","last_change":"2024-03-16T20:45:05.428065+0000","last_active":"2024-03-16T20:45:06.459997+0000","last_peered":"2024-03-16T20:45:06.459997+0000","last_clean":"2024-03-16T20:45:06.459997+0000","last_became_active":"2024-03-16T20:45:05.427498+0000","last_became_peered":"2024-03-16T20:45:05.427498+0000","last_unstale":"2024-03-16T20:45:06.459997+0000","last_undegraded":"2024-03-16T20:45:06.459997+0000","last_fullsized":"2024-03-16T20:45:06.459997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:40:17.773165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410546+0000","last_change":"2024-03-16T20:45:05.418257+0000","last_active":"2024-03-16T20:45:06.410546+0000","last_peered":"2024-03-16T20:45:06.410546+0000","last_clean":"2024-03-16T20:45:06.410546+0000","last_became_active":"2024-03-16T20:45:05.416660+0000","last_became_peered":"2024-03-16T20:45:05.416660+0000","last_unstale":"2024-03-16T20:45:06.410546+0000","last_undegraded":"2024-03-16T20:45:06.410546+0000","last_fullsized":"2024-03-16T20:45:06.410546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:29:10.023600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.418359+0000","last_change":"2024-03-16T20:45:05.410148+0000","last_active":"2024-03-16T20:45:05.418359+0000","last_peered":"2024-03-16T20:45:05.418359+0000","last_clean":"2024-03-16T20:45:05.418359+0000","last_became_active":"2024-03-16T20:45:05.409853+0000","last_became_peered":"2024-03-16T20:45:05.409853+0000","last_unstale":"2024-03-16T20:45:05.418359+0000","last_undegraded":"2024-03-16T20:45:05.418359+0000","last_fullsized":"2024-03-16T20:45:05.418359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:58.407763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412946+0000","last_change":"2024-03-16T20:45:05.419996+0000","last_active":"2024-03-16T20:45:06.412946+0000","last_peered":"2024-03-16T20:45:06.412946+0000","last_clean":"2024-03-16T20:45:06.412946+0000","last_became_active":"2024-03-16T20:45:05.417478+0000","last_became_peered":"2024-03-16T20:45:05.417478+0000","last_unstale":"2024-03-16T20:45:06.412946+0000","last_undegraded":"2024-03-16T20:45:06.412946+0000","last_fullsized":"2024-03-16T20:45:06.412946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:26:54.432519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459267+0000","last_change":"2024-03-16T20:45:05.422236+0000","last_active":"2024-03-16T20:45:06.459267+0000","last_peered":"2024-03-16T20:45:06.459267+0000","last_clean":"2024-03-16T20:45:06.459267+0000","last_became_active":"2024-03-16T20:45:05.421698+0000","last_became_peered":"2024-03-16T20:45:05.421698+0000","last_unstale":"2024-03-16T20:45:06.459267+0000","last_undegraded":"2024-03-16T20:45:06.459267+0000","last_fullsized":"2024-03-16T20:45:06.459267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:54.931097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412907+0000","last_change":"2024-03-16T20:45:05.419865+0000","last_active":"2024-03-16T20:45:06.412907+0000","last_peered":"2024-03-16T20:45:06.412907+0000","last_clean":"2024-03-16T20:45:06.412907+0000","last_became_active":"2024-03-16T20:45:05.419476+0000","last_became_peered":"2024-03-16T20:45:05.419476+0000","last_unstale":"2024-03-16T20:45:06.412907+0000","last_undegraded":"2024-03-16T20:45:06.412907+0000","last_fullsized":"2024-03-16T20:45:06.412907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:50:22.920748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.458816+0000","last_change":"2024-03-16T20:45:05.421986+0000","last_active":"2024-03-16T20:45:06.458816+0000","last_peered":"2024-03-16T20:45:06.458816+0000","last_clean":"2024-03-16T20:45:06.458816+0000","last_became_active":"2024-03-16T20:45:05.421125+0000","last_became_peered":"2024-03-16T20:45:05.421125+0000","last_unstale":"2024-03-16T20:45:06.458816+0000","last_undegraded":"2024-03-16T20:45:06.458816+0000","last_fullsized":"2024-03-16T20:45:06.458816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:26:54.432519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.414440+0000","last_change":"2024-03-16T20:45:05.402329+0000","last_active":"2024-03-16T20:45:05.414440+0000","last_peered":"2024-03-16T20:45:05.414440+0000","last_clean":"2024-03-16T20:45:05.414440+0000","last_became_active":"2024-03-16T20:45:05.400677+0000","last_became_peered":"2024-03-16T20:45:05.400677+0000","last_unstale":"2024-03-16T20:45:05.414440+0000","last_undegraded":"2024-03-16T20:45:05.414440+0000","last_fullsized":"2024-03-16T20:45:05.414440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:03:04.245122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.460167+0000","last_change":"2024-03-16T20:45:05.425819+0000","last_active":"2024-03-16T20:45:06.460167+0000","last_peered":"2024-03-16T20:45:06.460167+0000","last_clean":"2024-03-16T20:45:06.460167+0000","last_became_active":"2024-03-16T20:45:05.425238+0000","last_became_peered":"2024-03-16T20:45:05.425238+0000","last_unstale":"2024-03-16T20:45:06.460167+0000","last_undegraded":"2024-03-16T20:45:06.460167+0000","last_fullsized":"2024-03-16T20:45:06.460167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:04:42.305935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412787+0000","last_change":"2024-03-16T20:45:05.419949+0000","last_active":"2024-03-16T20:45:06.412787+0000","last_peered":"2024-03-16T20:45:06.412787+0000","last_clean":"2024-03-16T20:45:06.412787+0000","last_became_active":"2024-03-16T20:45:05.419670+0000","last_became_peered":"2024-03-16T20:45:05.419670+0000","last_unstale":"2024-03-16T20:45:06.412787+0000","last_undegraded":"2024-03-16T20:45:06.412787+0000","last_fullsized":"2024-03-16T20:45:06.412787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:35:07.902704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412340+0000","last_change":"2024-03-16T20:45:05.424035+0000","last_active":"2024-03-16T20:45:06.412340+0000","last_peered":"2024-03-16T20:45:06.412340+0000","last_clean":"2024-03-16T20:45:06.412340+0000","last_became_active":"2024-03-16T20:45:05.423657+0000","last_became_peered":"2024-03-16T20:45:05.423657+0000","last_unstale":"2024-03-16T20:45:06.412340+0000","last_undegraded":"2024-03-16T20:45:06.412340+0000","last_fullsized":"2024-03-16T20:45:06.412340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:03:04.229071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.436044+0000","last_change":"2024-03-16T20:45:05.427007+0000","last_active":"2024-03-16T20:45:05.436044+0000","last_peered":"2024-03-16T20:45:05.436044+0000","last_clean":"2024-03-16T20:45:05.436044+0000","last_became_active":"2024-03-16T20:45:05.426482+0000","last_became_peered":"2024-03-16T20:45:05.426482+0000","last_unstale":"2024-03-16T20:45:05.436044+0000","last_undegraded":"2024-03-16T20:45:05.436044+0000","last_fullsized":"2024-03-16T20:45:05.436044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:28:49.680196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.422180+0000","last_change":"2024-03-16T20:45:05.418866+0000","last_active":"2024-03-16T20:45:05.422180+0000","last_peered":"2024-03-16T20:45:05.422180+0000","last_clean":"2024-03-16T20:45:05.422180+0000","last_became_active":"2024-03-16T20:45:05.418458+0000","last_became_peered":"2024-03-16T20:45:05.418458+0000","last_unstale":"2024-03-16T20:45:05.422180+0000","last_undegraded":"2024-03-16T20:45:05.422180+0000","last_fullsized":"2024-03-16T20:45:05.422180+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:16:59.281226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.418320+0000","last_change":"2024-03-16T20:45:05.411615+0000","last_active":"2024-03-16T20:45:05.418320+0000","last_peered":"2024-03-16T20:45:05.418320+0000","last_clean":"2024-03-16T20:45:05.418320+0000","last_became_active":"2024-03-16T20:45:05.410789+0000","last_became_peered":"2024-03-16T20:45:05.410789+0000","last_unstale":"2024-03-16T20:45:05.418320+0000","last_undegraded":"2024-03-16T20:45:05.418320+0000","last_fullsized":"2024-03-16T20:45:05.418320+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:50:17.556011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410139+0000","last_change":"2024-03-16T20:45:05.411051+0000","last_active":"2024-03-16T20:45:06.410139+0000","last_peered":"2024-03-16T20:45:06.410139+0000","last_clean":"2024-03-16T20:45:06.410139+0000","last_became_active":"2024-03-16T20:45:05.409805+0000","last_became_peered":"2024-03-16T20:45:05.409805+0000","last_unstale":"2024-03-16T20:45:06.410139+0000","last_undegraded":"2024-03-16T20:45:06.410139+0000","last_fullsized":"2024-03-16T20:45:06.410139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:26:51.023909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.411744+0000","last_change":"2024-03-16T20:45:05.413320+0000","last_active":"2024-03-16T20:45:06.411744+0000","last_peered":"2024-03-16T20:45:06.411744+0000","last_clean":"2024-03-16T20:45:06.411744+0000","last_became_active":"2024-03-16T20:45:05.412473+0000","last_became_peered":"2024-03-16T20:45:05.412473+0000","last_unstale":"2024-03-16T20:45:06.411744+0000","last_undegraded":"2024-03-16T20:45:06.411744+0000","last_fullsized":"2024-03-16T20:45:06.411744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:35:07.902704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.414480+0000","last_change":"2024-03-16T20:45:05.421924+0000","last_active":"2024-03-16T20:45:06.414480+0000","last_peered":"2024-03-16T20:45:06.414480+0000","last_clean":"2024-03-16T20:45:06.414480+0000","last_became_active":"2024-03-16T20:45:05.421632+0000","last_became_peered":"2024-03-16T20:45:05.421632+0000","last_unstale":"2024-03-16T20:45:06.414480+0000","last_undegraded":"2024-03-16T20:45:06.414480+0000","last_fullsized":"2024-03-16T20:45:06.414480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:18:48.237612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.429671+0000","last_change":"2024-03-16T20:45:05.424802+0000","last_active":"2024-03-16T20:45:05.429671+0000","last_peered":"2024-03-16T20:45:05.429671+0000","last_clean":"2024-03-16T20:45:05.429671+0000","last_became_active":"2024-03-16T20:45:05.424388+0000","last_became_peered":"2024-03-16T20:45:05.424388+0000","last_unstale":"2024-03-16T20:45:05.429671+0000","last_undegraded":"2024-03-16T20:45:05.429671+0000","last_fullsized":"2024-03-16T20:45:05.429671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:23:33.686287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.460115+0000","last_change":"2024-03-16T20:45:05.418605+0000","last_active":"2024-03-16T20:45:06.460115+0000","last_peered":"2024-03-16T20:45:06.460115+0000","last_clean":"2024-03-16T20:45:06.460115+0000","last_became_active":"2024-03-16T20:45:05.418293+0000","last_became_peered":"2024-03-16T20:45:05.418293+0000","last_unstale":"2024-03-16T20:45:06.460115+0000","last_undegraded":"2024-03-16T20:45:06.460115+0000","last_fullsized":"2024-03-16T20:45:06.460115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:57:45.943288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412374+0000","last_change":"2024-03-16T20:45:05.422968+0000","last_active":"2024-03-16T20:45:06.412374+0000","last_peered":"2024-03-16T20:45:06.412374+0000","last_clean":"2024-03-16T20:45:06.412374+0000","last_became_active":"2024-03-16T20:45:05.422612+0000","last_became_peered":"2024-03-16T20:45:05.422612+0000","last_unstale":"2024-03-16T20:45:06.412374+0000","last_undegraded":"2024-03-16T20:45:06.412374+0000","last_fullsized":"2024-03-16T20:45:06.412374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:26:51.023909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.429954+0000","last_change":"2024-03-16T20:45:05.426370+0000","last_active":"2024-03-16T20:45:05.429954+0000","last_peered":"2024-03-16T20:45:05.429954+0000","last_clean":"2024-03-16T20:45:05.429954+0000","last_became_active":"2024-03-16T20:45:05.426035+0000","last_became_peered":"2024-03-16T20:45:05.426035+0000","last_unstale":"2024-03-16T20:45:05.429954+0000","last_undegraded":"2024-03-16T20:45:05.429954+0000","last_fullsized":"2024-03-16T20:45:05.429954+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:16:40.202022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413393+0000","last_change":"2024-03-16T20:45:05.410178+0000","last_active":"2024-03-16T20:45:06.413393+0000","last_peered":"2024-03-16T20:45:06.413393+0000","last_clean":"2024-03-16T20:45:06.413393+0000","last_became_active":"2024-03-16T20:45:05.409867+0000","last_became_peered":"2024-03-16T20:45:05.409867+0000","last_unstale":"2024-03-16T20:45:06.413393+0000","last_undegraded":"2024-03-16T20:45:06.413393+0000","last_fullsized":"2024-03-16T20:45:06.413393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:54.931097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.417027+0000","last_change":"2024-03-16T20:45:05.413326+0000","last_active":"2024-03-16T20:45:05.417027+0000","last_peered":"2024-03-16T20:45:05.417027+0000","last_clean":"2024-03-16T20:45:05.417027+0000","last_became_active":"2024-03-16T20:45:05.412843+0000","last_became_peered":"2024-03-16T20:45:05.412843+0000","last_unstale":"2024-03-16T20:45:05.417027+0000","last_undegraded":"2024-03-16T20:45:05.417027+0000","last_fullsized":"2024-03-16T20:45:05.417027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:23:33.686287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412143+0000","last_change":"2024-03-16T20:45:05.404852+0000","last_active":"2024-03-16T20:45:06.412143+0000","last_peered":"2024-03-16T20:45:06.412143+0000","last_clean":"2024-03-16T20:45:06.412143+0000","last_became_active":"2024-03-16T20:45:05.403728+0000","last_became_peered":"2024-03-16T20:45:05.403728+0000","last_unstale":"2024-03-16T20:45:06.412143+0000","last_undegraded":"2024-03-16T20:45:06.412143+0000","last_fullsized":"2024-03-16T20:45:06.412143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:29:10.023600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.418093+0000","last_change":"2024-03-16T20:45:05.414436+0000","last_active":"2024-03-16T20:45:05.418093+0000","last_peered":"2024-03-16T20:45:05.418093+0000","last_clean":"2024-03-16T20:45:05.418093+0000","last_became_active":"2024-03-16T20:45:05.413903+0000","last_became_peered":"2024-03-16T20:45:05.413903+0000","last_unstale":"2024-03-16T20:45:05.418093+0000","last_undegraded":"2024-03-16T20:45:05.418093+0000","last_fullsized":"2024-03-16T20:45:05.418093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:20:09.822409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413175+0000","last_change":"2024-03-16T20:45:05.423967+0000","last_active":"2024-03-16T20:45:06.413175+0000","last_peered":"2024-03-16T20:45:06.413175+0000","last_clean":"2024-03-16T20:45:06.413175+0000","last_became_active":"2024-03-16T20:45:05.423691+0000","last_became_peered":"2024-03-16T20:45:05.423691+0000","last_unstale":"2024-03-16T20:45:06.413175+0000","last_undegraded":"2024-03-16T20:45:06.413175+0000","last_fullsized":"2024-03-16T20:45:06.413175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:29:10.023600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410152+0000","last_change":"2024-03-16T20:45:05.418695+0000","last_active":"2024-03-16T20:45:06.410152+0000","last_peered":"2024-03-16T20:45:06.410152+0000","last_clean":"2024-03-16T20:45:06.410152+0000","last_became_active":"2024-03-16T20:45:05.415980+0000","last_became_peered":"2024-03-16T20:45:05.415980+0000","last_unstale":"2024-03-16T20:45:06.410152+0000","last_undegraded":"2024-03-16T20:45:06.410152+0000","last_fullsized":"2024-03-16T20:45:06.410152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:54:36.635249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.420167+0000","last_change":"2024-03-16T20:45:05.414732+0000","last_active":"2024-03-16T20:45:05.420167+0000","last_peered":"2024-03-16T20:45:05.420167+0000","last_clean":"2024-03-16T20:45:05.420167+0000","last_became_active":"2024-03-16T20:45:05.411367+0000","last_became_peered":"2024-03-16T20:45:05.411367+0000","last_unstale":"2024-03-16T20:45:05.420167+0000","last_undegraded":"2024-03-16T20:45:05.420167+0000","last_fullsized":"2024-03-16T20:45:05.420167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:16:40.202022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.427828+0000","last_change":"2024-03-16T20:45:05.422746+0000","last_active":"2024-03-16T20:45:05.427828+0000","last_peered":"2024-03-16T20:45:05.427828+0000","last_clean":"2024-03-16T20:45:05.427828+0000","last_became_active":"2024-03-16T20:45:05.422339+0000","last_became_peered":"2024-03-16T20:45:05.422339+0000","last_unstale":"2024-03-16T20:45:05.427828+0000","last_undegraded":"2024-03-16T20:45:05.427828+0000","last_fullsized":"2024-03-16T20:45:05.427828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:31:58.827839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413606+0000","last_change":"2024-03-16T20:45:05.419051+0000","last_active":"2024-03-16T20:45:06.413606+0000","last_peered":"2024-03-16T20:45:06.413606+0000","last_clean":"2024-03-16T20:45:06.413606+0000","last_became_active":"2024-03-16T20:45:05.417745+0000","last_became_peered":"2024-03-16T20:45:05.417745+0000","last_unstale":"2024-03-16T20:45:06.413606+0000","last_undegraded":"2024-03-16T20:45:06.413606+0000","last_fullsized":"2024-03-16T20:45:06.413606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:15:41.403418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.416748+0000","last_change":"2024-03-16T20:45:05.409514+0000","last_active":"2024-03-16T20:45:05.416748+0000","last_peered":"2024-03-16T20:45:05.416748+0000","last_clean":"2024-03-16T20:45:05.416748+0000","last_became_active":"2024-03-16T20:45:05.408695+0000","last_became_peered":"2024-03-16T20:45:05.408695+0000","last_unstale":"2024-03-16T20:45:05.416748+0000","last_undegraded":"2024-03-16T20:45:05.416748+0000","last_fullsized":"2024-03-16T20:45:05.416748+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:03:04.245122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.411788+0000","last_change":"2024-03-16T20:45:05.417983+0000","last_active":"2024-03-16T20:45:06.411788+0000","last_peered":"2024-03-16T20:45:06.411788+0000","last_clean":"2024-03-16T20:45:06.411788+0000","last_became_active":"2024-03-16T20:45:05.417669+0000","last_became_peered":"2024-03-16T20:45:05.417669+0000","last_unstale":"2024-03-16T20:45:06.411788+0000","last_undegraded":"2024-03-16T20:45:06.411788+0000","last_fullsized":"2024-03-16T20:45:06.411788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:15:27.440101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.411105+0000","last_change":"2024-03-16T20:45:05.407932+0000","last_active":"2024-03-16T20:45:05.411105+0000","last_peered":"2024-03-16T20:45:05.411105+0000","last_clean":"2024-03-16T20:45:05.411105+0000","last_became_active":"2024-03-16T20:45:05.407504+0000","last_became_peered":"2024-03-16T20:45:05.407504+0000","last_unstale":"2024-03-16T20:45:05.411105+0000","last_undegraded":"2024-03-16T20:45:05.411105+0000","last_fullsized":"2024-03-16T20:45:05.411105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:09:40.709188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.437724+0000","last_change":"2024-03-16T20:45:04.380160+0000","last_active":"2024-03-16T20:45:05.437724+0000","last_peered":"2024-03-16T20:45:05.437724+0000","last_clean":"2024-03-16T20:45:05.437724+0000","last_became_active":"2024-03-16T20:45:04.379622+0000","last_became_peered":"2024-03-16T20:45:04.379622+0000","last_unstale":"2024-03-16T20:45:05.437724+0000","last_undegraded":"2024-03-16T20:45:05.437724+0000","last_fullsized":"2024-03-16T20:45:05.437724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:57.399645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.425223+0000","last_change":"2024-03-16T20:45:05.421374+0000","last_active":"2024-03-16T20:45:05.425223+0000","last_peered":"2024-03-16T20:45:05.425223+0000","last_clean":"2024-03-16T20:45:05.425223+0000","last_became_active":"2024-03-16T20:45:05.420935+0000","last_became_peered":"2024-03-16T20:45:05.420935+0000","last_unstale":"2024-03-16T20:45:05.425223+0000","last_undegraded":"2024-03-16T20:45:05.425223+0000","last_fullsized":"2024-03-16T20:45:05.425223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:48:49.183385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412742+0000","last_change":"2024-03-16T20:45:04.389177+0000","last_active":"2024-03-16T20:45:06.412742+0000","last_peered":"2024-03-16T20:45:06.412742+0000","last_clean":"2024-03-16T20:45:06.412742+0000","last_became_active":"2024-03-16T20:45:04.388433+0000","last_became_peered":"2024-03-16T20:45:04.388433+0000","last_unstale":"2024-03-16T20:45:06.412742+0000","last_undegraded":"2024-03-16T20:45:06.412742+0000","last_fullsized":"2024-03-16T20:45:06.412742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:48:48.175266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.394852+0000","last_change":"2024-03-16T20:45:04.383850+0000","last_active":"2024-03-16T20:45:05.394852+0000","last_peered":"2024-03-16T20:45:05.394852+0000","last_clean":"2024-03-16T20:45:05.394852+0000","last_became_active":"2024-03-16T20:45:04.383588+0000","last_became_peered":"2024-03-16T20:45:04.383588+0000","last_unstale":"2024-03-16T20:45:05.394852+0000","last_undegraded":"2024-03-16T20:45:05.394852+0000","last_fullsized":"2024-03-16T20:45:05.394852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:50:16.547893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.420132+0000","last_change":"2024-03-16T20:45:05.414605+0000","last_active":"2024-03-16T20:45:05.420132+0000","last_peered":"2024-03-16T20:45:05.420132+0000","last_clean":"2024-03-16T20:45:05.420132+0000","last_became_active":"2024-03-16T20:45:05.414067+0000","last_became_peered":"2024-03-16T20:45:05.414067+0000","last_unstale":"2024-03-16T20:45:05.420132+0000","last_undegraded":"2024-03-16T20:45:05.420132+0000","last_fullsized":"2024-03-16T20:45:05.420132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:31:58.827839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412395+0000","last_change":"2024-03-16T20:45:04.387317+0000","last_active":"2024-03-16T20:45:06.412395+0000","last_peered":"2024-03-16T20:45:06.412395+0000","last_clean":"2024-03-16T20:45:06.412395+0000","last_became_active":"2024-03-16T20:45:04.386147+0000","last_became_peered":"2024-03-16T20:45:04.386147+0000","last_unstale":"2024-03-16T20:45:06.412395+0000","last_undegraded":"2024-03-16T20:45:06.412395+0000","last_fullsized":"2024-03-16T20:45:06.412395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:48:48.175266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410049+0000","last_change":"2024-03-16T20:45:05.419053+0000","last_active":"2024-03-16T20:45:06.410049+0000","last_peered":"2024-03-16T20:45:06.410049+0000","last_clean":"2024-03-16T20:45:06.410049+0000","last_became_active":"2024-03-16T20:45:05.418610+0000","last_became_peered":"2024-03-16T20:45:05.418610+0000","last_unstale":"2024-03-16T20:45:06.410049+0000","last_undegraded":"2024-03-16T20:45:06.410049+0000","last_fullsized":"2024-03-16T20:45:06.410049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:03:04.229071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.414172+0000","last_change":"2024-03-16T20:45:04.400713+0000","last_active":"2024-03-16T20:45:06.414172+0000","last_peered":"2024-03-16T20:45:06.414172+0000","last_clean":"2024-03-16T20:45:06.414172+0000","last_became_active":"2024-03-16T20:45:04.399907+0000","last_became_peered":"2024-03-16T20:45:04.399907+0000","last_unstale":"2024-03-16T20:45:06.414172+0000","last_undegraded":"2024-03-16T20:45:06.414172+0000","last_fullsized":"2024-03-16T20:45:06.414172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:34:16.171892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413409+0000","last_change":"2024-03-16T20:45:05.422773+0000","last_active":"2024-03-16T20:45:06.413409+0000","last_peered":"2024-03-16T20:45:06.413409+0000","last_clean":"2024-03-16T20:45:06.413409+0000","last_became_active":"2024-03-16T20:45:05.421962+0000","last_became_peered":"2024-03-16T20:45:05.421962+0000","last_unstale":"2024-03-16T20:45:06.413409+0000","last_undegraded":"2024-03-16T20:45:06.413409+0000","last_fullsized":"2024-03-16T20:45:06.413409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:11:19.673375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413369+0000","last_change":"2024-03-16T20:45:04.395629+0000","last_active":"2024-03-16T20:45:06.413369+0000","last_peered":"2024-03-16T20:45:06.413369+0000","last_clean":"2024-03-16T20:45:06.413369+0000","last_became_active":"2024-03-16T20:45:04.395107+0000","last_became_peered":"2024-03-16T20:45:04.395107+0000","last_unstale":"2024-03-16T20:45:06.413369+0000","last_undegraded":"2024-03-16T20:45:06.413369+0000","last_fullsized":"2024-03-16T20:45:06.413369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:03:03.237004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.424666+0000","last_change":"2024-03-16T20:45:05.421014+0000","last_active":"2024-03-16T20:45:05.424666+0000","last_peered":"2024-03-16T20:45:05.424666+0000","last_clean":"2024-03-16T20:45:05.424666+0000","last_became_active":"2024-03-16T20:45:05.420652+0000","last_became_peered":"2024-03-16T20:45:05.420652+0000","last_unstale":"2024-03-16T20:45:05.424666+0000","last_undegraded":"2024-03-16T20:45:05.424666+0000","last_fullsized":"2024-03-16T20:45:05.424666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:23:33.686287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410232+0000","last_change":"2024-03-16T20:45:04.396880+0000","last_active":"2024-03-16T20:45:06.410232+0000","last_peered":"2024-03-16T20:45:06.410232+0000","last_clean":"2024-03-16T20:45:06.410232+0000","last_became_active":"2024-03-16T20:45:04.396654+0000","last_became_peered":"2024-03-16T20:45:04.396654+0000","last_unstale":"2024-03-16T20:45:06.410232+0000","last_undegraded":"2024-03-16T20:45:06.410232+0000","last_fullsized":"2024-03-16T20:45:06.410232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:03:03.237004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.430969+0000","last_change":"2024-03-16T20:45:05.425535+0000","last_active":"2024-03-16T20:45:05.430969+0000","last_peered":"2024-03-16T20:45:05.430969+0000","last_clean":"2024-03-16T20:45:05.430969+0000","last_became_active":"2024-03-16T20:45:05.425236+0000","last_became_peered":"2024-03-16T20:45:05.425236+0000","last_unstale":"2024-03-16T20:45:05.430969+0000","last_undegraded":"2024-03-16T20:45:05.430969+0000","last_fullsized":"2024-03-16T20:45:05.430969+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:50:17.556011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412682+0000","last_change":"2024-03-16T20:45:04.384514+0000","last_active":"2024-03-16T20:45:06.412682+0000","last_peered":"2024-03-16T20:45:06.412682+0000","last_clean":"2024-03-16T20:45:06.412682+0000","last_became_active":"2024-03-16T20:45:04.379541+0000","last_became_peered":"2024-03-16T20:45:04.379541+0000","last_unstale":"2024-03-16T20:45:06.412682+0000","last_undegraded":"2024-03-16T20:45:06.412682+0000","last_fullsized":"2024-03-16T20:45:06.412682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:28:48.672077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459401+0000","last_change":"2024-03-16T20:45:05.416547+0000","last_active":"2024-03-16T20:45:06.459401+0000","last_peered":"2024-03-16T20:45:06.459401+0000","last_clean":"2024-03-16T20:45:06.459401+0000","last_became_active":"2024-03-16T20:45:05.416216+0000","last_became_peered":"2024-03-16T20:45:05.416216+0000","last_unstale":"2024-03-16T20:45:06.459401+0000","last_undegraded":"2024-03-16T20:45:06.459401+0000","last_fullsized":"2024-03-16T20:45:06.459401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:11:19.673375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.438777+0000","last_change":"2024-03-16T20:45:04.378488+0000","last_active":"2024-03-16T20:45:05.438777+0000","last_peered":"2024-03-16T20:45:05.438777+0000","last_clean":"2024-03-16T20:45:05.438777+0000","last_became_active":"2024-03-16T20:45:04.377819+0000","last_became_peered":"2024-03-16T20:45:04.377819+0000","last_unstale":"2024-03-16T20:45:05.438777+0000","last_undegraded":"2024-03-16T20:45:05.438777+0000","last_fullsized":"2024-03-16T20:45:05.438777+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:09:39.701069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.427190+0000","last_change":"2024-03-16T20:45:05.421867+0000","last_active":"2024-03-16T20:45:05.427190+0000","last_peered":"2024-03-16T20:45:05.427190+0000","last_clean":"2024-03-16T20:45:05.427190+0000","last_became_active":"2024-03-16T20:45:05.421377+0000","last_became_peered":"2024-03-16T20:45:05.421377+0000","last_unstale":"2024-03-16T20:45:05.427190+0000","last_undegraded":"2024-03-16T20:45:05.427190+0000","last_fullsized":"2024-03-16T20:45:05.427190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:20:09.822409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413633+0000","last_change":"2024-03-16T20:45:04.401205+0000","last_active":"2024-03-16T20:45:06.413633+0000","last_peered":"2024-03-16T20:45:06.413633+0000","last_clean":"2024-03-16T20:45:06.413633+0000","last_became_active":"2024-03-16T20:45:04.396416+0000","last_became_peered":"2024-03-16T20:45:04.396416+0000","last_unstale":"2024-03-16T20:45:06.413633+0000","last_undegraded":"2024-03-16T20:45:06.413633+0000","last_fullsized":"2024-03-16T20:45:06.413633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:28:48.672077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459532+0000","last_change":"2024-03-16T20:45:05.421976+0000","last_active":"2024-03-16T20:45:06.459532+0000","last_peered":"2024-03-16T20:45:06.459532+0000","last_clean":"2024-03-16T20:45:06.459532+0000","last_became_active":"2024-03-16T20:45:05.421370+0000","last_became_peered":"2024-03-16T20:45:05.421370+0000","last_unstale":"2024-03-16T20:45:06.459532+0000","last_undegraded":"2024-03-16T20:45:06.459532+0000","last_fullsized":"2024-03-16T20:45:06.459532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:15:41.403418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.411884+0000","last_change":"2024-03-16T20:45:04.389332+0000","last_active":"2024-03-16T20:45:06.411884+0000","last_peered":"2024-03-16T20:45:06.411884+0000","last_clean":"2024-03-16T20:45:06.411884+0000","last_became_active":"2024-03-16T20:45:04.388661+0000","last_became_peered":"2024-03-16T20:45:04.388661+0000","last_unstale":"2024-03-16T20:45:06.411884+0000","last_undegraded":"2024-03-16T20:45:06.411884+0000","last_fullsized":"2024-03-16T20:45:06.411884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:23:32.678168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412210+0000","last_change":"2024-03-16T20:45:05.418621+0000","last_active":"2024-03-16T20:45:06.412210+0000","last_peered":"2024-03-16T20:45:06.412210+0000","last_clean":"2024-03-16T20:45:06.412210+0000","last_became_active":"2024-03-16T20:45:05.418292+0000","last_became_peered":"2024-03-16T20:45:05.418292+0000","last_unstale":"2024-03-16T20:45:06.412210+0000","last_undegraded":"2024-03-16T20:45:06.412210+0000","last_fullsized":"2024-03-16T20:45:06.412210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:22:42.961272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.440095+0000","last_change":"2024-03-16T20:45:04.359171+0000","last_active":"2024-03-16T20:45:05.440095+0000","last_peered":"2024-03-16T20:45:05.440095+0000","last_clean":"2024-03-16T20:45:05.440095+0000","last_became_active":"2024-03-16T20:45:04.358863+0000","last_became_peered":"2024-03-16T20:45:04.358863+0000","last_unstale":"2024-03-16T20:45:05.440095+0000","last_undegraded":"2024-03-16T20:45:05.440095+0000","last_fullsized":"2024-03-16T20:45:05.440095+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:28:48.672077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459047+0000","last_change":"2024-03-16T20:45:05.417061+0000","last_active":"2024-03-16T20:45:06.459047+0000","last_peered":"2024-03-16T20:45:06.459047+0000","last_clean":"2024-03-16T20:45:06.459047+0000","last_became_active":"2024-03-16T20:45:05.416454+0000","last_became_peered":"2024-03-16T20:45:05.416454+0000","last_unstale":"2024-03-16T20:45:06.459047+0000","last_undegraded":"2024-03-16T20:45:06.459047+0000","last_fullsized":"2024-03-16T20:45:06.459047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:55:30.648219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.436121+0000","last_change":"2024-03-16T20:45:04.376055+0000","last_active":"2024-03-16T20:45:05.436121+0000","last_peered":"2024-03-16T20:45:05.436121+0000","last_clean":"2024-03-16T20:45:05.436121+0000","last_became_active":"2024-03-16T20:45:04.375272+0000","last_became_peered":"2024-03-16T20:45:04.375272+0000","last_unstale":"2024-03-16T20:45:05.436121+0000","last_undegraded":"2024-03-16T20:45:05.436121+0000","last_fullsized":"2024-03-16T20:45:05.436121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:52:31.895539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459647+0000","last_change":"2024-03-16T20:45:05.420665+0000","last_active":"2024-03-16T20:45:06.459647+0000","last_peered":"2024-03-16T20:45:06.459647+0000","last_clean":"2024-03-16T20:45:06.459647+0000","last_became_active":"2024-03-16T20:45:05.420363+0000","last_became_peered":"2024-03-16T20:45:05.420363+0000","last_unstale":"2024-03-16T20:45:06.459647+0000","last_undegraded":"2024-03-16T20:45:06.459647+0000","last_fullsized":"2024-03-16T20:45:06.459647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:59:02.640185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.440676+0000","last_change":"2024-03-16T20:45:04.363877+0000","last_active":"2024-03-16T20:45:05.440676+0000","last_peered":"2024-03-16T20:45:05.440676+0000","last_clean":"2024-03-16T20:45:05.440676+0000","last_became_active":"2024-03-16T20:45:04.363180+0000","last_became_peered":"2024-03-16T20:45:04.363180+0000","last_unstale":"2024-03-16T20:45:05.440676+0000","last_undegraded":"2024-03-16T20:45:05.440676+0000","last_fullsized":"2024-03-16T20:45:05.440676+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:57.399645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.460378+0000","last_change":"2024-03-16T20:45:05.429416+0000","last_active":"2024-03-16T20:45:06.460378+0000","last_peered":"2024-03-16T20:45:06.460378+0000","last_clean":"2024-03-16T20:45:06.460378+0000","last_became_active":"2024-03-16T20:45:05.428766+0000","last_became_peered":"2024-03-16T20:45:05.428766+0000","last_unstale":"2024-03-16T20:45:06.460378+0000","last_undegraded":"2024-03-16T20:45:06.460378+0000","last_fullsized":"2024-03-16T20:45:06.460378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:45:30.989735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410331+0000","last_change":"2024-03-16T20:45:04.355156+0000","last_active":"2024-03-16T20:45:06.410331+0000","last_peered":"2024-03-16T20:45:06.410331+0000","last_clean":"2024-03-16T20:45:06.410331+0000","last_became_active":"2024-03-16T20:45:04.354621+0000","last_became_peered":"2024-03-16T20:45:04.354621+0000","last_unstale":"2024-03-16T20:45:06.410331+0000","last_undegraded":"2024-03-16T20:45:06.410331+0000","last_fullsized":"2024-03-16T20:45:06.410331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:31:57.819721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413605+0000","last_change":"2024-03-16T20:45:05.419655+0000","last_active":"2024-03-16T20:45:06.413605+0000","last_peered":"2024-03-16T20:45:06.413605+0000","last_clean":"2024-03-16T20:45:06.413605+0000","last_became_active":"2024-03-16T20:45:05.417669+0000","last_became_peered":"2024-03-16T20:45:05.417669+0000","last_unstale":"2024-03-16T20:45:06.413605+0000","last_undegraded":"2024-03-16T20:45:06.413605+0000","last_fullsized":"2024-03-16T20:45:06.413605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:59:02.640185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.397516+0000","last_change":"2024-03-16T20:45:04.401781+0000","last_active":"2024-03-16T20:45:05.397516+0000","last_peered":"2024-03-16T20:45:05.397516+0000","last_clean":"2024-03-16T20:45:05.397516+0000","last_became_active":"2024-03-16T20:45:04.401563+0000","last_became_peered":"2024-03-16T20:45:04.401563+0000","last_unstale":"2024-03-16T20:45:05.397516+0000","last_undegraded":"2024-03-16T20:45:05.397516+0000","last_fullsized":"2024-03-16T20:45:05.397516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:34:16.171892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459736+0000","last_change":"2024-03-16T20:45:05.413949+0000","last_active":"2024-03-16T20:45:06.459736+0000","last_peered":"2024-03-16T20:45:06.459736+0000","last_clean":"2024-03-16T20:45:06.459736+0000","last_became_active":"2024-03-16T20:45:05.413609+0000","last_became_peered":"2024-03-16T20:45:05.413609+0000","last_unstale":"2024-03-16T20:45:06.459736+0000","last_undegraded":"2024-03-16T20:45:06.459736+0000","last_fullsized":"2024-03-16T20:45:06.459736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:18:48.237612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.458981+0000","last_change":"2024-03-16T20:44:54.293418+0000","last_active":"2024-03-16T20:45:06.458981+0000","last_peered":"2024-03-16T20:45:06.458981+0000","last_clean":"2024-03-16T20:45:06.458981+0000","last_became_active":"2024-03-16T20:44:54.291347+0000","last_became_peered":"2024-03-16T20:44:54.291347+0000","last_unstale":"2024-03-16T20:45:06.458981+0000","last_undegraded":"2024-03-16T20:45:06.458981+0000","last_fullsized":"2024-03-16T20:45:06.458981+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_clean_scrub_stamp":"2024-03-16T20:44:53.077639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:29:18.318494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412202+0000","last_change":"2024-03-16T20:45:04.394662+0000","last_active":"2024-03-16T20:45:06.412202+0000","last_peered":"2024-03-16T20:45:06.412202+0000","last_clean":"2024-03-16T20:45:06.412202+0000","last_became_active":"2024-03-16T20:45:04.394119+0000","last_became_peered":"2024-03-16T20:45:04.394119+0000","last_unstale":"2024-03-16T20:45:06.412202+0000","last_undegraded":"2024-03-16T20:45:06.412202+0000","last_fullsized":"2024-03-16T20:45:06.412202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:34:16.171892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.417991+0000","last_change":"2024-03-16T20:45:05.413913+0000","last_active":"2024-03-16T20:45:05.417991+0000","last_peered":"2024-03-16T20:45:05.417991+0000","last_clean":"2024-03-16T20:45:05.417991+0000","last_became_active":"2024-03-16T20:45:05.412780+0000","last_became_peered":"2024-03-16T20:45:05.412780+0000","last_unstale":"2024-03-16T20:45:05.417991+0000","last_undegraded":"2024-03-16T20:45:05.417991+0000","last_fullsized":"2024-03-16T20:45:05.417991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:48:49.183385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.458865+0000","last_change":"2024-03-16T20:45:04.380237+0000","last_active":"2024-03-16T20:45:06.458865+0000","last_peered":"2024-03-16T20:45:06.458865+0000","last_clean":"2024-03-16T20:45:06.458865+0000","last_became_active":"2024-03-16T20:45:04.379086+0000","last_became_peered":"2024-03-16T20:45:04.379086+0000","last_unstale":"2024-03-16T20:45:06.458865+0000","last_undegraded":"2024-03-16T20:45:06.458865+0000","last_fullsized":"2024-03-16T20:45:06.458865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:23:32.678168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.418535+0000","last_change":"2024-03-16T20:45:05.414361+0000","last_active":"2024-03-16T20:45:05.418535+0000","last_peered":"2024-03-16T20:45:05.418535+0000","last_clean":"2024-03-16T20:45:05.418535+0000","last_became_active":"2024-03-16T20:45:05.411900+0000","last_became_peered":"2024-03-16T20:45:05.411900+0000","last_unstale":"2024-03-16T20:45:05.418535+0000","last_undegraded":"2024-03-16T20:45:05.418535+0000","last_fullsized":"2024-03-16T20:45:05.418535+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:34:17.180010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.395647+0000","last_change":"2024-03-16T20:45:04.363289+0000","last_active":"2024-03-16T20:45:05.395647+0000","last_peered":"2024-03-16T20:45:05.395647+0000","last_clean":"2024-03-16T20:45:05.395647+0000","last_became_active":"2024-03-16T20:45:04.362919+0000","last_became_peered":"2024-03-16T20:45:04.362919+0000","last_unstale":"2024-03-16T20:45:05.395647+0000","last_undegraded":"2024-03-16T20:45:05.395647+0000","last_fullsized":"2024-03-16T20:45:05.395647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:50:16.547893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.409757+0000","last_change":"2024-03-16T20:45:05.408984+0000","last_active":"2024-03-16T20:45:06.409757+0000","last_peered":"2024-03-16T20:45:06.409757+0000","last_clean":"2024-03-16T20:45:06.409757+0000","last_became_active":"2024-03-16T20:45:05.408648+0000","last_became_peered":"2024-03-16T20:45:05.408648+0000","last_unstale":"2024-03-16T20:45:06.409757+0000","last_undegraded":"2024-03-16T20:45:06.409757+0000","last_fullsized":"2024-03-16T20:45:06.409757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:22:42.961272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413903+0000","last_change":"2024-03-16T20:45:04.394583+0000","last_active":"2024-03-16T20:45:06.413903+0000","last_peered":"2024-03-16T20:45:06.413903+0000","last_clean":"2024-03-16T20:45:06.413903+0000","last_became_active":"2024-03-16T20:45:04.394016+0000","last_became_peered":"2024-03-16T20:45:04.394016+0000","last_unstale":"2024-03-16T20:45:06.413903+0000","last_undegraded":"2024-03-16T20:45:06.413903+0000","last_fullsized":"2024-03-16T20:45:06.413903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:23:32.678168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.432318+0000","last_change":"2024-03-16T20:45:05.425183+0000","last_active":"2024-03-16T20:45:05.432318+0000","last_peered":"2024-03-16T20:45:05.432318+0000","last_clean":"2024-03-16T20:45:05.432318+0000","last_became_active":"2024-03-16T20:45:05.424862+0000","last_became_peered":"2024-03-16T20:45:05.424862+0000","last_unstale":"2024-03-16T20:45:05.432318+0000","last_undegraded":"2024-03-16T20:45:05.432318+0000","last_fullsized":"2024-03-16T20:45:05.432318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:58.407763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410481+0000","last_change":"2024-03-16T20:45:04.354012+0000","last_active":"2024-03-16T20:45:06.410481+0000","last_peered":"2024-03-16T20:45:06.410481+0000","last_clean":"2024-03-16T20:45:06.410481+0000","last_became_active":"2024-03-16T20:45:04.353589+0000","last_became_peered":"2024-03-16T20:45:04.353589+0000","last_unstale":"2024-03-16T20:45:06.410481+0000","last_undegraded":"2024-03-16T20:45:06.410481+0000","last_fullsized":"2024-03-16T20:45:06.410481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:48:48.175266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.415027+0000","last_change":"2024-03-16T20:45:05.417774+0000","last_active":"2024-03-16T20:45:06.415027+0000","last_peered":"2024-03-16T20:45:06.415027+0000","last_clean":"2024-03-16T20:45:06.415027+0000","last_became_active":"2024-03-16T20:45:05.417404+0000","last_became_peered":"2024-03-16T20:45:05.417404+0000","last_unstale":"2024-03-16T20:45:06.415027+0000","last_undegraded":"2024-03-16T20:45:06.415027+0000","last_fullsized":"2024-03-16T20:45:06.415027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:57:45.943288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459976+0000","last_change":"2024-03-16T20:45:04.381406+0000","last_active":"2024-03-16T20:45:06.459976+0000","last_peered":"2024-03-16T20:45:06.459976+0000","last_clean":"2024-03-16T20:45:06.459976+0000","last_became_active":"2024-03-16T20:45:04.380608+0000","last_became_peered":"2024-03-16T20:45:04.380608+0000","last_unstale":"2024-03-16T20:45:06.459976+0000","last_undegraded":"2024-03-16T20:45:06.459976+0000","last_fullsized":"2024-03-16T20:45:06.459976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:16:39.193904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410306+0000","last_change":"2024-03-16T20:45:05.408949+0000","last_active":"2024-03-16T20:45:06.410306+0000","last_peered":"2024-03-16T20:45:06.410306+0000","last_clean":"2024-03-16T20:45:06.410306+0000","last_became_active":"2024-03-16T20:45:05.408619+0000","last_became_peered":"2024-03-16T20:45:05.408619+0000","last_unstale":"2024-03-16T20:45:06.410306+0000","last_undegraded":"2024-03-16T20:45:06.410306+0000","last_fullsized":"2024-03-16T20:45:06.410306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:35:07.902704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410456+0000","last_change":"2024-03-16T20:45:04.383772+0000","last_active":"2024-03-16T20:45:06.410456+0000","last_peered":"2024-03-16T20:45:06.410456+0000","last_clean":"2024-03-16T20:45:06.410456+0000","last_became_active":"2024-03-16T20:45:04.383050+0000","last_became_peered":"2024-03-16T20:45:04.383050+0000","last_unstale":"2024-03-16T20:45:06.410456+0000","last_undegraded":"2024-03-16T20:45:06.410456+0000","last_fullsized":"2024-03-16T20:45:06.410456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:20:08.814290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.422503+0000","last_change":"2024-03-16T20:45:05.419648+0000","last_active":"2024-03-16T20:45:05.422503+0000","last_peered":"2024-03-16T20:45:05.422503+0000","last_clean":"2024-03-16T20:45:05.422503+0000","last_became_active":"2024-03-16T20:45:05.419358+0000","last_became_peered":"2024-03-16T20:45:05.419358+0000","last_unstale":"2024-03-16T20:45:05.422503+0000","last_undegraded":"2024-03-16T20:45:05.422503+0000","last_fullsized":"2024-03-16T20:45:05.422503+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:06:46.922690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.414177+0000","last_change":"2024-03-16T20:45:04.382826+0000","last_active":"2024-03-16T20:45:06.414177+0000","last_peered":"2024-03-16T20:45:06.414177+0000","last_clean":"2024-03-16T20:45:06.414177+0000","last_became_active":"2024-03-16T20:45:04.381824+0000","last_became_peered":"2024-03-16T20:45:04.381824+0000","last_unstale":"2024-03-16T20:45:06.414177+0000","last_undegraded":"2024-03-16T20:45:06.414177+0000","last_fullsized":"2024-03-16T20:45:06.414177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:03:03.237004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.418612+0000","last_change":"2024-03-16T20:45:05.415644+0000","last_active":"2024-03-16T20:45:05.418612+0000","last_peered":"2024-03-16T20:45:05.418612+0000","last_clean":"2024-03-16T20:45:05.418612+0000","last_became_active":"2024-03-16T20:45:05.415294+0000","last_became_peered":"2024-03-16T20:45:05.415294+0000","last_unstale":"2024-03-16T20:45:05.418612+0000","last_undegraded":"2024-03-16T20:45:05.418612+0000","last_fullsized":"2024-03-16T20:45:05.418612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:03:04.245122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.458835+0000","last_change":"2024-03-16T20:45:04.380383+0000","last_active":"2024-03-16T20:45:06.458835+0000","last_peered":"2024-03-16T20:45:06.458835+0000","last_clean":"2024-03-16T20:45:06.458835+0000","last_became_active":"2024-03-16T20:45:04.379121+0000","last_became_peered":"2024-03-16T20:45:04.379121+0000","last_unstale":"2024-03-16T20:45:06.458835+0000","last_undegraded":"2024-03-16T20:45:06.458835+0000","last_fullsized":"2024-03-16T20:45:06.458835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:03:03.237004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.412642+0000","last_change":"2024-03-16T20:45:05.408820+0000","last_active":"2024-03-16T20:45:05.412642+0000","last_peered":"2024-03-16T20:45:05.412642+0000","last_clean":"2024-03-16T20:45:05.412642+0000","last_became_active":"2024-03-16T20:45:05.408337+0000","last_became_peered":"2024-03-16T20:45:05.408337+0000","last_unstale":"2024-03-16T20:45:05.412642+0000","last_undegraded":"2024-03-16T20:45:05.412642+0000","last_fullsized":"2024-03-16T20:45:05.412642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:43:47.642524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410303+0000","last_change":"2024-03-16T20:45:04.379953+0000","last_active":"2024-03-16T20:45:06.410303+0000","last_peered":"2024-03-16T20:45:06.410303+0000","last_clean":"2024-03-16T20:45:06.410303+0000","last_became_active":"2024-03-16T20:45:04.379610+0000","last_became_peered":"2024-03-16T20:45:04.379610+0000","last_unstale":"2024-03-16T20:45:06.410303+0000","last_undegraded":"2024-03-16T20:45:06.410303+0000","last_fullsized":"2024-03-16T20:45:06.410303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:23:32.678168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.414416+0000","last_change":"2024-03-16T20:45:05.401685+0000","last_active":"2024-03-16T20:45:05.414416+0000","last_peered":"2024-03-16T20:45:05.414416+0000","last_clean":"2024-03-16T20:45:05.414416+0000","last_became_active":"2024-03-16T20:45:05.399878+0000","last_became_peered":"2024-03-16T20:45:05.399878+0000","last_unstale":"2024-03-16T20:45:05.414416+0000","last_undegraded":"2024-03-16T20:45:05.414416+0000","last_fullsized":"2024-03-16T20:45:05.414416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:34:17.180010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410121+0000","last_change":"2024-03-16T20:45:04.359852+0000","last_active":"2024-03-16T20:45:06.410121+0000","last_peered":"2024-03-16T20:45:06.410121+0000","last_clean":"2024-03-16T20:45:06.410121+0000","last_became_active":"2024-03-16T20:45:04.359557+0000","last_became_peered":"2024-03-16T20:45:04.359557+0000","last_unstale":"2024-03-16T20:45:06.410121+0000","last_undegraded":"2024-03-16T20:45:06.410121+0000","last_fullsized":"2024-03-16T20:45:06.410121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:34:16.171892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.432755+0000","last_change":"2024-03-16T20:45:05.422880+0000","last_active":"2024-03-16T20:45:05.432755+0000","last_peered":"2024-03-16T20:45:05.432755+0000","last_clean":"2024-03-16T20:45:05.432755+0000","last_became_active":"2024-03-16T20:45:05.422585+0000","last_became_peered":"2024-03-16T20:45:05.422585+0000","last_unstale":"2024-03-16T20:45:05.432755+0000","last_undegraded":"2024-03-16T20:45:05.432755+0000","last_fullsized":"2024-03-16T20:45:05.432755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:08:02.898870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459415+0000","last_change":"2024-03-16T20:45:04.382087+0000","last_active":"2024-03-16T20:45:06.459415+0000","last_peered":"2024-03-16T20:45:06.459415+0000","last_clean":"2024-03-16T20:45:06.459415+0000","last_became_active":"2024-03-16T20:45:04.381331+0000","last_became_peered":"2024-03-16T20:45:04.381331+0000","last_unstale":"2024-03-16T20:45:06.459415+0000","last_undegraded":"2024-03-16T20:45:06.459415+0000","last_fullsized":"2024-03-16T20:45:06.459415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:31:57.819721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412636+0000","last_change":"2024-03-16T20:45:05.416889+0000","last_active":"2024-03-16T20:45:06.412636+0000","last_peered":"2024-03-16T20:45:06.412636+0000","last_clean":"2024-03-16T20:45:06.412636+0000","last_became_active":"2024-03-16T20:45:05.416560+0000","last_became_peered":"2024-03-16T20:45:05.416560+0000","last_unstale":"2024-03-16T20:45:06.412636+0000","last_undegraded":"2024-03-16T20:45:06.412636+0000","last_fullsized":"2024-03-16T20:45:06.412636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:54:36.635249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.414635+0000","last_change":"2024-03-16T20:45:05.403182+0000","last_active":"2024-03-16T20:45:06.414635+0000","last_peered":"2024-03-16T20:45:06.414635+0000","last_clean":"2024-03-16T20:45:06.414635+0000","last_became_active":"2024-03-16T20:45:05.401656+0000","last_became_peered":"2024-03-16T20:45:05.401656+0000","last_unstale":"2024-03-16T20:45:06.414635+0000","last_undegraded":"2024-03-16T20:45:06.414635+0000","last_fullsized":"2024-03-16T20:45:06.414635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:40:17.773165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459476+0000","last_change":"2024-03-16T20:45:04.402308+0000","last_active":"2024-03-16T20:45:06.459476+0000","last_peered":"2024-03-16T20:45:06.459476+0000","last_clean":"2024-03-16T20:45:06.459476+0000","last_became_active":"2024-03-16T20:45:04.401750+0000","last_became_peered":"2024-03-16T20:45:04.401750+0000","last_unstale":"2024-03-16T20:45:06.459476+0000","last_undegraded":"2024-03-16T20:45:06.459476+0000","last_fullsized":"2024-03-16T20:45:06.459476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:48:48.175266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413317+0000","last_change":"2024-03-16T20:45:05.412664+0000","last_active":"2024-03-16T20:45:06.413317+0000","last_peered":"2024-03-16T20:45:06.413317+0000","last_clean":"2024-03-16T20:45:06.413317+0000","last_became_active":"2024-03-16T20:45:05.412302+0000","last_became_peered":"2024-03-16T20:45:05.412302+0000","last_unstale":"2024-03-16T20:45:06.413317+0000","last_undegraded":"2024-03-16T20:45:06.413317+0000","last_fullsized":"2024-03-16T20:45:06.413317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:55:30.648219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459690+0000","last_change":"2024-03-16T20:45:04.387378+0000","last_active":"2024-03-16T20:45:06.459690+0000","last_peered":"2024-03-16T20:45:06.459690+0000","last_clean":"2024-03-16T20:45:06.459690+0000","last_became_active":"2024-03-16T20:45:04.386602+0000","last_became_peered":"2024-03-16T20:45:04.386602+0000","last_unstale":"2024-03-16T20:45:06.459690+0000","last_undegraded":"2024-03-16T20:45:06.459690+0000","last_fullsized":"2024-03-16T20:45:06.459690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:20:08.814290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.418357+0000","last_change":"2024-03-16T20:45:05.413222+0000","last_active":"2024-03-16T20:45:05.418357+0000","last_peered":"2024-03-16T20:45:05.418357+0000","last_clean":"2024-03-16T20:45:05.418357+0000","last_became_active":"2024-03-16T20:45:05.412620+0000","last_became_peered":"2024-03-16T20:45:05.412620+0000","last_unstale":"2024-03-16T20:45:05.418357+0000","last_undegraded":"2024-03-16T20:45:05.418357+0000","last_fullsized":"2024-03-16T20:45:05.418357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:28:49.680196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.437739+0000","last_change":"2024-03-16T20:45:04.362033+0000","last_active":"2024-03-16T20:45:05.437739+0000","last_peered":"2024-03-16T20:45:05.437739+0000","last_clean":"2024-03-16T20:45:05.437739+0000","last_became_active":"2024-03-16T20:45:04.361100+0000","last_became_peered":"2024-03-16T20:45:04.361100+0000","last_unstale":"2024-03-16T20:45:05.437739+0000","last_undegraded":"2024-03-16T20:45:05.437739+0000","last_fullsized":"2024-03-16T20:45:05.437739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:28:48.672077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.432875+0000","last_change":"2024-03-16T20:45:05.425640+0000","last_active":"2024-03-16T20:45:05.432875+0000","last_peered":"2024-03-16T20:45:05.432875+0000","last_clean":"2024-03-16T20:45:05.432875+0000","last_became_active":"2024-03-16T20:45:05.424729+0000","last_became_peered":"2024-03-16T20:45:05.424729+0000","last_unstale":"2024-03-16T20:45:05.432875+0000","last_undegraded":"2024-03-16T20:45:05.432875+0000","last_fullsized":"2024-03-16T20:45:05.432875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:23:41.936524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-03-16T20:45:05.438704+0000","last_change":"2024-03-16T20:45:04.380444+0000","last_active":"2024-03-16T20:45:05.438704+0000","last_peered":"2024-03-16T20:45:05.438704+0000","last_clean":"2024-03-16T20:45:05.438704+0000","last_became_active":"2024-03-16T20:45:04.380164+0000","last_became_peered":"2024-03-16T20:45:04.380164+0000","last_unstale":"2024-03-16T20:45:05.438704+0000","last_undegraded":"2024-03-16T20:45:05.438704+0000","last_fullsized":"2024-03-16T20:45:05.438704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:43:46.634405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31964,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339876,"statfs":{"total":96636764160,"available":96604033024,"internally_reserved":0,"allocated":802816,"data_stored":660255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":70917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":208896,"data_stored":69395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":69431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":69305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":69465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":660311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T20:45:08.782 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-16T20:45:08.782 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-16T20:45:08.782 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-03-16T20:45:08.782 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-03-16T20:45:08.782 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-03-16T20:45:08.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.928+0000 7f6d1caa4640 1 Processor -- start 2024-03-16T20:45:08.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.928+0000 7f6d1caa4640 1 -- start start 2024-03-16T20:45:08.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d1caa4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6d1813d8d0 0x7f6d1806b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d1caa4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d18071580 0x7f6d1814dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d1caa4640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d180b4c50 con 0x7f6d18146a50 2024-03-16T20:45:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d1caa4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d180b2c30 con 0x7f6d1813d8d0 2024-03-16T20:45:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d1caa4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d18146ee0 con 0x7f6d18071580 2024-03-16T20:45:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d1659c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6d1813d8d0 0x7f6d1806b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d1659c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6d1813d8d0 0x7f6d1806b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41162/0 (socket says 172.21.15.122:41162) 2024-03-16T20:45:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d15d9b640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f6d18146a50 0x7f6d18143460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58502/0 (socket says 172.21.15.122:58502) 2024-03-16T20:45:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d1659c640 1 -- 172.21.15.122:0/167244091 learned_addr learned my addr 172.21.15.122:0/167244091 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d16d9d640 1 --2- 172.21.15.122:0/167244091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d18071580 0x7f6d1814dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d1659c640 1 -- 172.21.15.122:0/167244091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d18071580 msgr2=0x7f6d1814dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d1659c640 1 --2- 172.21.15.122:0/167244091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d18071580 0x7f6d1814dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d1659c640 1 -- 172.21.15.122:0/167244091 >> v1:172.21.15.122:6789/0 conn(0x7f6d18146a50 legacy=0x7f6d18143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d1659c640 1 -- 172.21.15.122:0/167244091 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d181471c0 con 0x7f6d1813d8d0 2024-03-16T20:45:08.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d1559a640 1 -- 172.21.15.122:0/167244091 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3503413817 0 0) 0x7f6d180b4c50 con 0x7f6d18146a50 2024-03-16T20:45:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d1659c640 1 --2- 172.21.15.122:0/167244091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d1813d8d0 0x7f6d1806b820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800aa10 tx=0x7f6d0801d930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57cb0f124992f9a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d1559a640 1 -- 172.21.15.122:0/167244091 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080a68a0 con 0x7f6d1813d8d0 2024-03-16T20:45:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d1559a640 1 -- 172.21.15.122:0/167244091 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d080a6a00 con 0x7f6d1813d8d0 2024-03-16T20:45:08.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.932+0000 7f6d1559a640 1 -- 172.21.15.122:0/167244091 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080a6d90 con 0x7f6d1813d8d0 2024-03-16T20:45:08.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 -- 172.21.15.122:0/167244091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d1813d8d0 msgr2=0x7f6d1806b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 --2- 172.21.15.122:0/167244091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d1813d8d0 0x7f6d1806b820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800aa10 tx=0x7f6d0801d930 comp rx=0 tx=0).stop 2024-03-16T20:45:08.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 -- 172.21.15.122:0/167244091 shutdown_connections 2024-03-16T20:45:08.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 --2- 172.21.15.122:0/167244091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d18071580 0x7f6d1814dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 --2- 172.21.15.122:0/167244091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d1813d8d0 0x7f6d1806b820 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 -- 172.21.15.122:0/167244091 >> 172.21.15.122:0/167244091 conn(0x7f6d180b8860 msgr2=0x7f6d180b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:08.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 -- 172.21.15.122:0/167244091 shutdown_connections 2024-03-16T20:45:08.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 -- 172.21.15.122:0/167244091 wait complete. 2024-03-16T20:45:08.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 Processor -- start 2024-03-16T20:45:08.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 -- start start 2024-03-16T20:45:08.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d18071580 0x7f6d1813b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:08.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d1813d8d0 0x7f6d181e2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:08.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d18138b70 0x7f6d181e34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:08.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d18146020 con 0x7f6d18071580 2024-03-16T20:45:08.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d18144be0 con 0x7f6d18138b70 2024-03-16T20:45:08.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d18144ec0 con 0x7f6d1813d8d0 2024-03-16T20:45:08.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d18071580 0x7f6d1813b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:08.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d18071580 0x7f6d1813b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41168/0 (socket says 172.21.15.122:41168) 2024-03-16T20:45:08.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1659c640 1 -- 172.21.15.122:0/579917140 learned_addr learned my addr 172.21.15.122:0/579917140 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:08.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1659c640 1 -- 172.21.15.122:0/579917140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d1813d8d0 msgr2=0x7f6d181e2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1659c640 1 --2- 172.21.15.122:0/579917140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d1813d8d0 0x7f6d181e2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1659c640 1 -- 172.21.15.122:0/579917140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d18138b70 msgr2=0x7f6d181e34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:08.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1659c640 1 --2- 172.21.15.122:0/579917140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d18138b70 0x7f6d181e34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:08.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1659c640 1 -- 172.21.15.122:0/579917140 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d0800a670 con 0x7f6d18071580 2024-03-16T20:45:08.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1659c640 1 --2- 172.21.15.122:0/579917140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d18071580 0x7f6d1813b370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800a1c0 tx=0x7f6d0801a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:08.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d06ffd640 1 -- 172.21.15.122:0/579917140 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080ae950 con 0x7f6d18071580 2024-03-16T20:45:08.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d06ffd640 1 -- 172.21.15.122:0/579917140 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d080aeab0 con 0x7f6d18071580 2024-03-16T20:45:08.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 -- 172.21.15.122:0/579917140 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d18146290 con 0x7f6d18071580 2024-03-16T20:45:08.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d06ffd640 1 -- 172.21.15.122:0/579917140 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080aee40 con 0x7f6d18071580 2024-03-16T20:45:08.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 -- 172.21.15.122:0/579917140 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d18146720 con 0x7f6d18071580 2024-03-16T20:45:08.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.936+0000 7f6d1caa4640 1 -- 172.21.15.122:0/579917140 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d18071d10 con 0x7f6d18071580 2024-03-16T20:45:08.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.940+0000 7f6d06ffd640 1 -- 172.21.15.122:0/579917140 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f6d080b63f0 con 0x7f6d18071580 2024-03-16T20:45:08.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.940+0000 7f6d06ffd640 1 --2- 172.21.15.122:0/579917140 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6cf8066e40 0x7f6cf8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:08.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.940+0000 7f6d06ffd640 1 -- 172.21.15.122:0/579917140 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d0811ee70 con 0x7f6d18071580 2024-03-16T20:45:08.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.940+0000 7f6d15d9b640 1 --2- 172.21.15.122:0/579917140 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6cf8066e40 0x7f6cf8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:08.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.940+0000 7f6d15d9b640 1 --2- 172.21.15.122:0/579917140 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6cf8066e40 0x7f6cf8069300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c0011f0 tx=0x7f6d0c007930 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:08.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:08.944+0000 7f6d06ffd640 1 -- 172.21.15.122:0/579917140 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6d080ebe10 con 0x7f6d18071580 2024-03-16T20:45:09.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.124+0000 7f6d1caa4640 1 -- 172.21.15.122:0/579917140 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f6d18144a40 con 0x7f6d18071580 2024-03-16T20:45:09.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.128+0000 7f6d06ffd640 1 -- 172.21.15.122:0/579917140 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+165202 (secure 0 0 0) 0x7f6d18144a40 con 0x7f6d18071580 2024-03-16T20:45:09.137 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:09.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.132+0000 7f6d1caa4640 1 -- 172.21.15.122:0/579917140 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6cf8066e40 msgr2=0x7f6cf8069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.132+0000 7f6d1caa4640 1 --2- 172.21.15.122:0/579917140 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6cf8066e40 0x7f6cf8069300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c0011f0 tx=0x7f6d0c007930 comp rx=0 tx=0).stop 2024-03-16T20:45:09.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.132+0000 7f6d1caa4640 1 -- 172.21.15.122:0/579917140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d18071580 msgr2=0x7f6d1813b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.132+0000 7f6d1caa4640 1 --2- 172.21.15.122:0/579917140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d18071580 0x7f6d1813b370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6d0800a1c0 tx=0x7f6d0801a6d0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.132+0000 7f6d1caa4640 1 -- 172.21.15.122:0/579917140 shutdown_connections 2024-03-16T20:45:09.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.132+0000 7f6d1caa4640 1 --2- 172.21.15.122:0/579917140 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6cf8066e40 0x7f6cf8069300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.132+0000 7f6d1caa4640 1 --2- 172.21.15.122:0/579917140 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6d18138b70 0x7f6d181e34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.132+0000 7f6d1caa4640 1 --2- 172.21.15.122:0/579917140 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6d1813d8d0 0x7f6d181e2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.132+0000 7f6d1caa4640 1 --2- 172.21.15.122:0/579917140 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6d18071580 0x7f6d1813b370 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.132+0000 7f6d1caa4640 1 -- 172.21.15.122:0/579917140 >> 172.21.15.122:0/579917140 conn(0x7f6d180b8860 msgr2=0x7f6d180b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.132+0000 7f6d1caa4640 1 -- 172.21.15.122:0/579917140 shutdown_connections 2024-03-16T20:45:09.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.132+0000 7f6d1caa4640 1 -- 172.21.15.122:0/579917140 wait complete. 2024-03-16T20:45:09.154 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6832","nonce":3072163791},{"type":"v1","addr":"172.21.15.195:6833","nonce":3072163791}]},"active_addr":"172.21.15.195:6833/3072163791","active_change":"2024-03-16T20:44:44.565687+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4101,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":2251765048}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":3234245761}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":1462266855}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.195:0","nonce":1973250247}]}]} 2024-03-16T20:45:09.157 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-03-16T20:45:09.157 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-03-16T20:45:09.157 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:09.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.268+0000 7f8e1ddd4640 1 Processor -- start 2024-03-16T20:45:09.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.268+0000 7f8e1ddd4640 1 -- start start 2024-03-16T20:45:09.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.268+0000 7f8e1ddd4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f8e18144e10 0x7f8e18148ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.268+0000 7f8e1ddd4640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e180bb080 con 0x7f8e1813f240 2024-03-16T20:45:09.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.268+0000 7f8e1ddd4640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e180b1300 con 0x7f8e180baca0 2024-03-16T20:45:09.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.268+0000 7f8e1ddd4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e180b06b0 con 0x7f8e18144e10 2024-03-16T20:45:09.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.268+0000 7f8e17fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f8e18144e10 0x7f8e18148ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.268+0000 7f8e17fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f8e18144e10 0x7f8e18148ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41178/0 (socket says 172.21.15.122:41178) 2024-03-16T20:45:09.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.268+0000 7f8e17fff640 1 -- 172.21.15.122:0/693423950 learned_addr learned my addr 172.21.15.122:0/693423950 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:09.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.268+0000 7f8e0ffff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f8e1813f240 0x7f8e1814d780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58504/0 (socket says 172.21.15.122:58504) 2024-03-16T20:45:09.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.268+0000 7f8e17fff640 1 -- 172.21.15.122:0/693423950 >> v1:172.21.15.195:6789/0 conn(0x7f8e180baca0 legacy=0x7f8e1813eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:09.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.268+0000 7f8e17fff640 1 -- 172.21.15.122:0/693423950 >> v1:172.21.15.122:6789/0 conn(0x7f8e1813f240 legacy=0x7f8e1814d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.268+0000 7f8e17fff640 1 -- 172.21.15.122:0/693423950 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e18148400 con 0x7f8e18144e10 2024-03-16T20:45:09.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.268+0000 7f8e17fff640 1 --2- 172.21.15.122:0/693423950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e18144e10 0x7f8e18148ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009e50 tx=0x7f8e1001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6685fca233039e3c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.268+0000 7f8e16ffd640 1 -- 172.21.15.122:0/693423950 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100a56d0 con 0x7f8e18144e10 2024-03-16T20:45:09.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.268+0000 7f8e16ffd640 1 -- 172.21.15.122:0/693423950 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e100a5830 con 0x7f8e18144e10 2024-03-16T20:45:09.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.268+0000 7f8e16ffd640 1 -- 172.21.15.122:0/693423950 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100a5bc0 con 0x7f8e18144e10 2024-03-16T20:45:09.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 -- 172.21.15.122:0/693423950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e18144e10 msgr2=0x7f8e18148ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 --2- 172.21.15.122:0/693423950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e18144e10 0x7f8e18148ec0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009e50 tx=0x7f8e1001c920 comp rx=0 tx=0).stop 2024-03-16T20:45:09.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 -- 172.21.15.122:0/693423950 shutdown_connections 2024-03-16T20:45:09.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 --2- 172.21.15.122:0/693423950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e18144e10 0x7f8e18148ec0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 -- 172.21.15.122:0/693423950 >> 172.21.15.122:0/693423950 conn(0x7f8e1806ee40 msgr2=0x7f8e1806f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:09.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 -- 172.21.15.122:0/693423950 shutdown_connections 2024-03-16T20:45:09.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 -- 172.21.15.122:0/693423950 wait complete. 2024-03-16T20:45:09.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 Processor -- start 2024-03-16T20:45:09.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 -- start start 2024-03-16T20:45:09.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e180baca0 0x7f8e181401d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e1813f240 0x7f8e1813f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e18144e10 0x7f8e181ddf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e18140d20 con 0x7f8e180baca0 2024-03-16T20:45:09.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e18140fc0 con 0x7f8e1813f240 2024-03-16T20:45:09.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e18142380 con 0x7f8e18144e10 2024-03-16T20:45:09.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e180baca0 0x7f8e181401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e180baca0 0x7f8e181401d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41184/0 (socket says 172.21.15.122:41184) 2024-03-16T20:45:09.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e177fe640 1 -- 172.21.15.122:0/1339752091 learned_addr learned my addr 172.21.15.122:0/1339752091 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:09.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e177fe640 1 -- 172.21.15.122:0/1339752091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e18144e10 msgr2=0x7f8e181ddf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e177fe640 1 --2- 172.21.15.122:0/1339752091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e18144e10 0x7f8e181ddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e177fe640 1 -- 172.21.15.122:0/1339752091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e1813f240 msgr2=0x7f8e1813f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e0ffff640 1 --2- 172.21.15.122:0/1339752091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e1813f240 0x7f8e1813f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e177fe640 1 --2- 172.21.15.122:0/1339752091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e1813f240 0x7f8e1813f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e177fe640 1 -- 172.21.15.122:0/1339752091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e10009b30 con 0x7f8e180baca0 2024-03-16T20:45:09.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e177fe640 1 --2- 172.21.15.122:0/1339752091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e180baca0 0x7f8e181401d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800c480 tx=0x7f8e0800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e14ff9640 1 -- 172.21.15.122:0/1339752091 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08012830 con 0x7f8e180baca0 2024-03-16T20:45:09.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e14ff9640 1 -- 172.21.15.122:0/1339752091 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e080129b0 con 0x7f8e180baca0 2024-03-16T20:45:09.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 -- 172.21.15.122:0/1339752091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e18150180 con 0x7f8e180baca0 2024-03-16T20:45:09.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e14ff9640 1 -- 172.21.15.122:0/1339752091 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e08012d60 con 0x7f8e180baca0 2024-03-16T20:45:09.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 -- 172.21.15.122:0/1339752091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1813fcd0 con 0x7f8e180baca0 2024-03-16T20:45:09.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e1ddd4640 1 -- 172.21.15.122:0/1339752091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e181503c0 con 0x7f8e180baca0 2024-03-16T20:45:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.272+0000 7f8e14ff9640 1 -- 172.21.15.122:0/1339752091 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f8e0801c460 con 0x7f8e180baca0 2024-03-16T20:45:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.276+0000 7f8e14ff9640 1 --2- 172.21.15.122:0/1339752091 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8dfc066e40 0x7f8dfc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.276+0000 7f8e14ff9640 1 -- 172.21.15.122:0/1339752091 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e080848f0 con 0x7f8e180baca0 2024-03-16T20:45:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.276+0000 7f8e0ffff640 1 --2- 172.21.15.122:0/1339752091 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8dfc066e40 0x7f8dfc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.276+0000 7f8e0ffff640 1 --2- 172.21.15.122:0/1339752091 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8dfc066e40 0x7f8dfc069300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8e180b4860 tx=0x7f8e00009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.280+0000 7f8e14ff9640 1 -- 172.21.15.122:0/1339752091 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8e08051770 con 0x7f8e180baca0 2024-03-16T20:45:09.333 INFO:tasks.ceph.mds.b.smithi195.stderr:2024-03-16T20:45:09.328+0000 7f31b2cc3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-16T20:45:09.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.428+0000 7f8e1ddd4640 1 -- 172.21.15.122:0/1339752091 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e180677b0 con 0x7f8e180baca0 2024-03-16T20:45:09.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.428+0000 7f8e14ff9640 1 -- 172.21.15.122:0/1339752091 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f8e08054e30 con 0x7f8e180baca0 2024-03-16T20:45:09.437 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:09.437 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:06.404479+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:09.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.432+0000 7f8e1ddd4640 1 -- 172.21.15.122:0/1339752091 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8dfc066e40 msgr2=0x7f8dfc069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.432+0000 7f8e1ddd4640 1 --2- 172.21.15.122:0/1339752091 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8dfc066e40 0x7f8dfc069300 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8e180b4860 tx=0x7f8e00009040 comp rx=0 tx=0).stop 2024-03-16T20:45:09.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.432+0000 7f8e1ddd4640 1 -- 172.21.15.122:0/1339752091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e180baca0 msgr2=0x7f8e181401d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.432+0000 7f8e1ddd4640 1 --2- 172.21.15.122:0/1339752091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e180baca0 0x7f8e181401d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800c480 tx=0x7f8e0800c940 comp rx=0 tx=0).stop 2024-03-16T20:45:09.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.432+0000 7f8e1ddd4640 1 -- 172.21.15.122:0/1339752091 shutdown_connections 2024-03-16T20:45:09.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.432+0000 7f8e1ddd4640 1 --2- 172.21.15.122:0/1339752091 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8dfc066e40 0x7f8dfc069300 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.432+0000 7f8e1ddd4640 1 --2- 172.21.15.122:0/1339752091 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e18144e10 0x7f8e181ddf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.432+0000 7f8e1ddd4640 1 --2- 172.21.15.122:0/1339752091 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e1813f240 0x7f8e1813f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.432+0000 7f8e1ddd4640 1 --2- 172.21.15.122:0/1339752091 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e180baca0 0x7f8e181401d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.432+0000 7f8e1ddd4640 1 -- 172.21.15.122:0/1339752091 >> 172.21.15.122:0/1339752091 conn(0x7f8e1806ee40 msgr2=0x7f8e180b8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:09.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.432+0000 7f8e1ddd4640 1 -- 172.21.15.122:0/1339752091 shutdown_connections 2024-03-16T20:45:09.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.432+0000 7f8e1ddd4640 1 -- 172.21.15.122:0/1339752091 wait complete. 2024-03-16T20:45:09.456 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-03-16T20:45:09.457 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:09.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.552+0000 7fadc7793640 1 Processor -- start 2024-03-16T20:45:09.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.556+0000 7fadc7793640 1 -- start start 2024-03-16T20:45:09.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.556+0000 7fadc7793640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fadc0072120 0x7fadc00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.556+0000 7fadc7793640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fadc013ed60 0x7fadc013f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.556+0000 7fadc7793640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc00b1050 con 0x7fadc0148430 2024-03-16T20:45:09.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.556+0000 7fadc7793640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc0072500 con 0x7fadc0072120 2024-03-16T20:45:09.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.556+0000 7fadc7793640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc00b38a0 con 0x7fadc013ed60 2024-03-16T20:45:09.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.556+0000 7fadc552f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fadc0072120 0x7fadc00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.556+0000 7fadc552f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fadc0072120 0x7fadc00afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41196/0 (socket says 172.21.15.122:41196) 2024-03-16T20:45:09.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.556+0000 7fadc552f640 1 -- 172.21.15.122:0/700363287 learned_addr learned my addr 172.21.15.122:0/700363287 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:09.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.556+0000 7fadc552f640 1 -- 172.21.15.122:0/700363287 >> v2:172.21.15.195:3300/0 conn(0x7fadc013ed60 msgr2=0x7fadc013f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:09.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.556+0000 7fadc552f640 1 --2- 172.21.15.122:0/700363287 >> v2:172.21.15.195:3300/0 conn(0x7fadc013ed60 0x7fadc013f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.556+0000 7fadc552f640 1 -- 172.21.15.122:0/700363287 >> v1:172.21.15.195:6789/0 conn(0x7fadc0148430 legacy=0x7fadc006af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:09.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.556+0000 7fadc552f640 1 -- 172.21.15.122:0/700363287 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc00b3b80 con 0x7fadc0072120 2024-03-16T20:45:09.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.556+0000 7fadc552f640 1 --2- 172.21.15.122:0/700363287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc0072120 0x7fadc00afb90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fadc006e840 tx=0x7fadb001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10090c77442b2eb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.556+0000 7fadb7fff640 1 -- 172.21.15.122:0/700363287 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb00ab600 con 0x7fadc0072120 2024-03-16T20:45:09.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.556+0000 7fadb7fff640 1 -- 172.21.15.122:0/700363287 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb00a7050 con 0x7fadc0072120 2024-03-16T20:45:09.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.556+0000 7fadb7fff640 1 -- 172.21.15.122:0/700363287 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb00ab980 con 0x7fadc0072120 2024-03-16T20:45:09.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 -- 172.21.15.122:0/700363287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc0072120 msgr2=0x7fadc00afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 --2- 172.21.15.122:0/700363287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc0072120 0x7fadc00afb90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fadc006e840 tx=0x7fadb001c450 comp rx=0 tx=0).stop 2024-03-16T20:45:09.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 -- 172.21.15.122:0/700363287 shutdown_connections 2024-03-16T20:45:09.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 --2- 172.21.15.122:0/700363287 >> v2:172.21.15.195:3300/0 conn(0x7fadc013ed60 0x7fadc013f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 --2- 172.21.15.122:0/700363287 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc0072120 0x7fadc00afb90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 -- 172.21.15.122:0/700363287 >> 172.21.15.122:0/700363287 conn(0x7fadc00b8860 msgr2=0x7fadc00b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:09.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 -- 172.21.15.122:0/700363287 shutdown_connections 2024-03-16T20:45:09.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 -- 172.21.15.122:0/700363287 wait complete. 2024-03-16T20:45:09.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 Processor -- start 2024-03-16T20:45:09.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 -- start start 2024-03-16T20:45:09.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc0072120 0x7fadc013d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadc013ed60 0x7fadc01363d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadc0148430 0x7fadc01e1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc0140510 con 0x7fadc0072120 2024-03-16T20:45:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc0149ad0 con 0x7fadc013ed60 2024-03-16T20:45:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc013fec0 con 0x7fadc0148430 2024-03-16T20:45:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc552f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc0072120 0x7fadc013d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc552f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc0072120 0x7fadc013d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41212/0 (socket says 172.21.15.122:41212) 2024-03-16T20:45:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc552f640 1 -- 172.21.15.122:0/749860306 learned_addr learned my addr 172.21.15.122:0/749860306 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc5d30640 1 --2- 172.21.15.122:0/749860306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadc0148430 0x7fadc01e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc4d2e640 1 --2- 172.21.15.122:0/749860306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadc013ed60 0x7fadc01363d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc552f640 1 -- 172.21.15.122:0/749860306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadc0148430 msgr2=0x7fadc01e1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc552f640 1 --2- 172.21.15.122:0/749860306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadc0148430 0x7fadc01e1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc552f640 1 -- 172.21.15.122:0/749860306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadc013ed60 msgr2=0x7fadc01363d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc552f640 1 --2- 172.21.15.122:0/749860306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadc013ed60 0x7fadc01363d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc552f640 1 -- 172.21.15.122:0/749860306 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb00094d0 con 0x7fadc0072120 2024-03-16T20:45:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc4d2e640 1 --2- 172.21.15.122:0/749860306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadc013ed60 0x7fadc01363d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc552f640 1 --2- 172.21.15.122:0/749860306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc0072120 0x7fadc013d840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fadb001c930 tx=0x7fadb00191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadb5ffb640 1 -- 172.21.15.122:0/749860306 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb00b6640 con 0x7fadc0072120 2024-03-16T20:45:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadb5ffb640 1 -- 172.21.15.122:0/749860306 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb00b67a0 con 0x7fadc0072120 2024-03-16T20:45:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 -- 172.21.15.122:0/749860306 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc01401a0 con 0x7fadc0072120 2024-03-16T20:45:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadb5ffb640 1 -- 172.21.15.122:0/749860306 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb00b6b30 con 0x7fadc0072120 2024-03-16T20:45:09.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 -- 172.21.15.122:0/749860306 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc00af730 con 0x7fadc0072120 2024-03-16T20:45:09.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.560+0000 7fadc7793640 1 -- 172.21.15.122:0/749860306 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc0140360 con 0x7fadc0072120 2024-03-16T20:45:09.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.564+0000 7fadb5ffb640 1 -- 172.21.15.122:0/749860306 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fadb00b2040 con 0x7fadc0072120 2024-03-16T20:45:09.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.564+0000 7fadb5ffb640 1 --2- 172.21.15.122:0/749860306 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fada4066e70 0x7fada4069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.564+0000 7fadb5ffb640 1 -- 172.21.15.122:0/749860306 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadb011eb50 con 0x7fadc0072120 2024-03-16T20:45:09.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.564+0000 7fadc4d2e640 1 --2- 172.21.15.122:0/749860306 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fada4066e70 0x7fada4069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.564+0000 7fadc4d2e640 1 --2- 172.21.15.122:0/749860306 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fada4066e70 0x7fada4069330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fadbc006fd0 tx=0x7fadbc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.568+0000 7fadb5ffb640 1 -- 172.21.15.122:0/749860306 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fadb00eb940 con 0x7fadc0072120 2024-03-16T20:45:09.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.712+0000 7fadc7793640 1 -- 172.21.15.122:0/749860306 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadc013fcf0 con 0x7fadc0072120 2024-03-16T20:45:09.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.716+0000 7fadb5ffb640 1 -- 172.21.15.122:0/749860306 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7fadb00ef000 con 0x7fadc0072120 2024-03-16T20:45:09.722 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:09.722 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:06.404479+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:09.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.716+0000 7fadc7793640 1 -- 172.21.15.122:0/749860306 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fada4066e70 msgr2=0x7fada4069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.716+0000 7fadc7793640 1 --2- 172.21.15.122:0/749860306 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fada4066e70 0x7fada4069330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fadbc006fd0 tx=0x7fadbc008040 comp rx=0 tx=0).stop 2024-03-16T20:45:09.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.716+0000 7fadc7793640 1 -- 172.21.15.122:0/749860306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc0072120 msgr2=0x7fadc013d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.716+0000 7fadc7793640 1 --2- 172.21.15.122:0/749860306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc0072120 0x7fadc013d840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fadb001c930 tx=0x7fadb00191f0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.716+0000 7fadc7793640 1 -- 172.21.15.122:0/749860306 shutdown_connections 2024-03-16T20:45:09.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.716+0000 7fadc7793640 1 --2- 172.21.15.122:0/749860306 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fada4066e70 0x7fada4069330 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.716+0000 7fadc7793640 1 --2- 172.21.15.122:0/749860306 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fadc0148430 0x7fadc01e1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.716+0000 7fadc7793640 1 --2- 172.21.15.122:0/749860306 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fadc013ed60 0x7fadc01363d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.716+0000 7fadc7793640 1 --2- 172.21.15.122:0/749860306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fadc0072120 0x7fadc013d840 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.716+0000 7fadc7793640 1 -- 172.21.15.122:0/749860306 >> 172.21.15.122:0/749860306 conn(0x7fadc00b8860 msgr2=0x7fadc0145660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:09.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.716+0000 7fadc7793640 1 -- 172.21.15.122:0/749860306 shutdown_connections 2024-03-16T20:45:09.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.716+0000 7fadc7793640 1 -- 172.21.15.122:0/749860306 wait complete. 2024-03-16T20:45:09.740 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-03-16T20:45:09.740 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-03-16T20:45:09.741 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-03-16T20:45:09.741 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-03-16T20:45:09.741 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-03-16T20:45:09.742 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-03-16T20:45:09.742 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-03-16T20:45:09.742 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-03-16T20:45:09.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7fe004e30640 1 Processor -- start 2024-03-16T20:45:09.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7fe004e30640 1 -- start start 2024-03-16T20:45:09.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7f99ae1e5640 1 Processor -- start 2024-03-16T20:45:09.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7ff81aa44640 1 Processor -- start 2024-03-16T20:45:09.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7fe004e30640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe0000b5190 0x7fe0001435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7fe004e30640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe0000b5570 0x7fe0001403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7fe004e30640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0000b59f0 con 0x7fe0000b8db0 2024-03-16T20:45:09.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7fe004e30640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe000143c50 con 0x7fe0000b5190 2024-03-16T20:45:09.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7fe004e30640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe000143f30 con 0x7fe0000b5570 2024-03-16T20:45:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7ff81aa44640 1 -- start start 2024-03-16T20:45:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7fdffe59c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe0000b5190 0x7fe0001435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7fdffe59c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe0000b5190 0x7fe0001435e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41228/0 (socket says 172.21.15.122:41228) 2024-03-16T20:45:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7fdffe59c640 1 -- 172.21.15.122:0/892447008 learned_addr learned my addr 172.21.15.122:0/892447008 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:09.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7ff81aa44640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff80c033920 0x7ff80c036540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7ff81aa44640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff80c033d00 0x7ff80c035560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7ff81aa44640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff80c036e00 con 0x7ff80c0c8600 2024-03-16T20:45:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7ff81aa44640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff80c036a80 con 0x7ff80c033920 2024-03-16T20:45:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7ff81aa44640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c029fb0 con 0x7ff80c033d00 2024-03-16T20:45:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7f99ae1e5640 1 -- start start 2024-03-16T20:45:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7f99ae1e5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99a0033950 0x7f99a0036570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7f99ae1e5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f99a00c8630 0x7f99a00d2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7f99ae1e5640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a0036e30 con 0x7f99a0033d30 2024-03-16T20:45:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7f99ae1e5640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a0036ab0 con 0x7f99a00c8630 2024-03-16T20:45:09.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7f99ae1e5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a0029fe0 con 0x7f99a0033950 2024-03-16T20:45:09.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fdffdd9b640 1 --2- 172.21.15.122:0/892447008 >> v2:172.21.15.195:3300/0 conn(0x7fe0000b5570 0x7fe0001403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7f029838b640 1 Processor -- start 2024-03-16T20:45:09.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.908+0000 7f029838b640 1 -- start start 2024-03-16T20:45:09.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f99a77fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99a0033950 0x7f99a0036570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f99a77fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99a0033950 0x7f99a0036570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:34160/0 (socket says 172.21.15.122:34160) 2024-03-16T20:45:09.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f99a77fe640 1 -- 172.21.15.122:0/770670872 learned_addr learned my addr 172.21.15.122:0/770670872 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:09.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f99a7fff640 1 --2- 172.21.15.122:0/770670872 >> v2:172.21.15.195:3300/0 conn(0x7f99a00c8630 0x7f99a00d2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7ff813fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff80c033920 0x7ff80c036540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.921 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fdffd59a640 1 -- 172.21.15.122:0/892447008 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3492718578 0 0) 0x7fe0000b59f0 con 0x7fe0000b8db0 2024-03-16T20:45:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7ff813fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff80c033920 0x7ff80c036540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41242/0 (socket says 172.21.15.122:41242) 2024-03-16T20:45:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7ff813fff640 1 -- 172.21.15.122:0/237079349 learned_addr learned my addr 172.21.15.122:0/237079349 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fdffd59a640 1 -- 172.21.15.122:0/892447008 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfe8003530 con 0x7fe0000b8db0 2024-03-16T20:45:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7ff8137fe640 1 --2- 172.21.15.122:0/237079349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff80c033d00 0x7ff80c035560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fab84c74640 1 Processor -- start 2024-03-16T20:45:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f99a77fe640 1 -- 172.21.15.122:0/770670872 >> v2:172.21.15.195:3300/0 conn(0x7f99a00c8630 msgr2=0x7f99a00d2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f99a77fe640 1 --2- 172.21.15.122:0/770670872 >> v2:172.21.15.195:3300/0 conn(0x7f99a00c8630 0x7f99a00d2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f99a77fe640 1 -- 172.21.15.122:0/770670872 >> v1:172.21.15.122:6789/0 conn(0x7f99a0033d30 legacy=0x7f99a0035590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fdffdd9b640 1 -- 172.21.15.122:0/892447008 >> v2:172.21.15.122:3300/0 conn(0x7fe0000b5190 msgr2=0x7fe0001435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fdffdd9b640 1 --2- 172.21.15.122:0/892447008 >> v2:172.21.15.122:3300/0 conn(0x7fe0000b5190 0x7fe0001435e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fdffdd9b640 1 -- 172.21.15.122:0/892447008 >> v1:172.21.15.195:6789/0 conn(0x7fe0000b8db0 legacy=0x7fe00006ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.923 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fdffdd9b640 1 -- 172.21.15.122:0/892447008 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00006c970 con 0x7fe0000b5570 2024-03-16T20:45:09.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f23d5fe6640 1 Processor -- start 2024-03-16T20:45:09.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f23d5fe6640 1 -- start start 2024-03-16T20:45:09.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f23cf7fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f23d00b4a40 0x7f23d00b7ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58542/0 (socket says 172.21.15.122:58542) 2024-03-16T20:45:09.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f23cf7fe640 1 -- 172.21.15.122:0/640065492 learned_addr learned my addr 172.21.15.122:0/640065492 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:09.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f23d5fe6640 1 --2- 172.21.15.122:0/640065492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23d0095180 0x7f23d00955a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f23d5fe6640 1 -- 172.21.15.122:0/640065492 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d0157d10 con 0x7f23d00b4a40 2024-03-16T20:45:09.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f23d5fe6640 1 -- 172.21.15.122:0/640065492 --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d006e1e0 con 0x7f23d0153ea0 2024-03-16T20:45:09.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f23d5fe6640 1 -- 172.21.15.122:0/640065492 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d01580f0 con 0x7f23d0095180 2024-03-16T20:45:09.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f23ceffd640 1 --2- 172.21.15.122:0/640065492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23d0095180 0x7f23d00955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fab84c74640 1 -- start start 2024-03-16T20:45:09.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fab84c74640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fab800b71d0 0x7fab800b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fab84c74640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab800b5010 con 0x7fab800b4be0 2024-03-16T20:45:09.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fab84c74640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab8014ea00 con 0x7fab800b8d00 2024-03-16T20:45:09.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fab84c74640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab800b58f0 con 0x7fab800b71d0 2024-03-16T20:45:09.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fab7f7fe640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fab800b8d00 0x7fab800b51e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.122:51234/0 (socket says 172.21.15.122:51234) 2024-03-16T20:45:09.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fab7f7fe640 1 -- 172.21.15.122:0/1549356665 learned_addr learned my addr 172.21.15.122:0/1549356665 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:09.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fab7effd640 1 --2- 172.21.15.122:0/1549356665 >> v2:172.21.15.122:3300/0 conn(0x7fab800b71d0 0x7fab800b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fab7effd640 1 -- 172.21.15.122:0/1549356665 >> v1:172.21.15.195:6789/0 conn(0x7fab800b8d00 legacy=0x7fab800b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fab7effd640 1 -- 172.21.15.122:0/1549356665 >> v1:172.21.15.122:6789/0 conn(0x7fab800b4be0 legacy=0x7fab8014eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:09.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fab7effd640 1 -- 172.21.15.122:0/1549356665 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab800b90e0 con 0x7fab800b71d0 2024-03-16T20:45:09.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7ff813fff640 1 -- 172.21.15.122:0/237079349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff80c033d00 msgr2=0x7ff80c035560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:09.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7ff813fff640 1 --2- 172.21.15.122:0/237079349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff80c033d00 0x7ff80c035560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7ff813fff640 1 -- 172.21.15.122:0/237079349 >> v1:172.21.15.122:6789/0 conn(0x7ff80c0c8600 legacy=0x7ff80c0cbb70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-03-16T20:45:09.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7ff813fff640 1 -- 172.21.15.122:0/237079349 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80c035ed0 con 0x7ff80c033920 2024-03-16T20:45:09.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f99a77fe640 1 -- 172.21.15.122:0/770670872 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a0035ca0 con 0x7f99a0033950 2024-03-16T20:45:09.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7fdffe59c640 1 --2- 172.21.15.122:0/892447008 >> v2:172.21.15.122:3300/0 conn(0x7fe0000b5190 0x7fe0001435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:09.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f029838b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f029014ef60 0x7f029014f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f029838b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f029014f900 0x7f02901563c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f029838b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02900b3520 con 0x7f029009ae30 2024-03-16T20:45:09.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f029838b640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02900b7a70 con 0x7f029014ef60 2024-03-16T20:45:09.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f029838b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02900b3770 con 0x7f029014f900 2024-03-16T20:45:09.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f0295926640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f029014ef60 0x7f029014f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f0295926640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f029014ef60 0x7f029014f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41258/0 (socket says 172.21.15.122:41258) 2024-03-16T20:45:09.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f0295926640 1 -- 172.21.15.122:0/1171526984 learned_addr learned my addr 172.21.15.122:0/1171526984 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:09.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f0296928640 1 --2- 172.21.15.122:0/1171526984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f029014f900 0x7f02901563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f0295926640 1 -- 172.21.15.122:0/1171526984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f029014f900 msgr2=0x7f02901563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:09.927 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f0295926640 1 --2- 172.21.15.122:0/1171526984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f029014f900 0x7f02901563c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f0295926640 1 -- 172.21.15.122:0/1171526984 >> v1:172.21.15.122:6789/0 conn(0x7f029009ae30 legacy=0x7f02900b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:09.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f0295926640 1 -- 172.21.15.122:0/1171526984 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029009b210 con 0x7f029014ef60 2024-03-16T20:45:09.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f23ce7fc640 1 -- 172.21.15.122:0/640065492 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468905291 0 0) 0x7f23d006e1e0 con 0x7f23d0153ea0 2024-03-16T20:45:09.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f23ce7fc640 1 -- 172.21.15.122:0/640065492 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23b8003530 con 0x7f23d0153ea0 2024-03-16T20:45:09.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f23ceffd640 1 -- 172.21.15.122:0/640065492 >> v1:172.21.15.195:6789/0 conn(0x7f23d0153ea0 legacy=0x7f23d0154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f23ce7fc640 1 -- 172.21.15.122:0/640065492 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425487541 0 0) 0x7f23d0157d10 con 0x7f23d00b4a40 2024-03-16T20:45:09.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f23ceffd640 1 -- 172.21.15.122:0/640065492 >> v1:172.21.15.122:6789/0 conn(0x7f23d00b4a40 legacy=0x7f23d00b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.928 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.912+0000 7f23ceffd640 1 -- 172.21.15.122:0/640065492 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d00b4e20 con 0x7f23d0095180 2024-03-16T20:45:09.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fdffdd9b640 1 --2- 172.21.15.122:0/892447008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0000b5570 0x7fe0001403f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe0000b5c00 tx=0x7fdff401c360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae20dc38a63351a2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f99a77fe640 1 --2- 172.21.15.122:0/770670872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99a0033950 0x7f99a0036570 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9994008870 tx=0x7f999401ea90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0b23620c5394afa server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fdffd59a640 1 -- 172.21.15.122:0/892447008 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff40ab600 con 0x7fe0000b5570 2024-03-16T20:45:09.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fdffd59a640 1 -- 172.21.15.122:0/892447008 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff40a7050 con 0x7fe0000b5570 2024-03-16T20:45:09.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fdffd59a640 1 -- 172.21.15.122:0/892447008 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff40ab980 con 0x7fe0000b5570 2024-03-16T20:45:09.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f99a67fc640 1 -- 172.21.15.122:0/770670872 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99940a7860 con 0x7f99a0033950 2024-03-16T20:45:09.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f99a67fc640 1 -- 172.21.15.122:0/770670872 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99940ab070 con 0x7f99a0033950 2024-03-16T20:45:09.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f99a67fc640 1 -- 172.21.15.122:0/770670872 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99940a7be0 con 0x7f99a0033950 2024-03-16T20:45:09.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f0295926640 1 --2- 172.21.15.122:0/1171526984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f029014ef60 0x7f029014f3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0280009870 tx=0x7f028001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e11da7a53f63768 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fe004e30640 1 -- 172.21.15.122:0/892447008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0000b5570 msgr2=0x7fe0001403f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fe004e30640 1 --2- 172.21.15.122:0/892447008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0000b5570 0x7fe0001403f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe0000b5c00 tx=0x7fdff401c360 comp rx=0 tx=0).stop 2024-03-16T20:45:09.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7ff813fff640 1 --2- 172.21.15.122:0/237079349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff80c033920 0x7ff80c036540 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff814069370 tx=0x7ff814099660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eead2a879ae75c1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fab7effd640 1 --2- 172.21.15.122:0/1549356665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab800b71d0 0x7fab800b46a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab800b92d0 tx=0x7fab7001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ed4c0a0482da19 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f99ae1e5640 1 -- 172.21.15.122:0/770670872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99a0033950 msgr2=0x7f99a0036570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f99ae1e5640 1 --2- 172.21.15.122:0/770670872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99a0033950 0x7f99a0036570 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9994008870 tx=0x7f999401ea90 comp rx=0 tx=0).stop 2024-03-16T20:45:09.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7ff812ffd640 1 -- 172.21.15.122:0/237079349 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81412c620 con 0x7ff80c033920 2024-03-16T20:45:09.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fab7e7fc640 1 -- 172.21.15.122:0/1549356665 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab700ab650 con 0x7fab800b71d0 2024-03-16T20:45:09.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fab7e7fc640 1 -- 172.21.15.122:0/1549356665 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab700ab7b0 con 0x7fab800b71d0 2024-03-16T20:45:09.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7ff812ffd640 1 -- 172.21.15.122:0/237079349 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81412c7c0 con 0x7ff80c033920 2024-03-16T20:45:09.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7ff812ffd640 1 -- 172.21.15.122:0/237079349 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81412cbf0 con 0x7ff80c033920 2024-03-16T20:45:09.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f99ae1e5640 1 -- 172.21.15.122:0/770670872 shutdown_connections 2024-03-16T20:45:09.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f99ae1e5640 1 --2- 172.21.15.122:0/770670872 >> v2:172.21.15.195:3300/0 conn(0x7f99a00c8630 0x7f99a00d2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f99ae1e5640 1 --2- 172.21.15.122:0/770670872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99a0033950 0x7f99a0036570 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f99ae1e5640 1 -- 172.21.15.122:0/770670872 >> 172.21.15.122:0/770670872 conn(0x7f99a003a890 msgr2=0x7f99a003aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:09.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f1e4be27640 1 Processor -- start 2024-03-16T20:45:09.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fe004e30640 1 -- 172.21.15.122:0/892447008 shutdown_connections 2024-03-16T20:45:09.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fe004e30640 1 --2- 172.21.15.122:0/892447008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0000b5570 0x7fe0001403f0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe0000b5c00 tx=0x7fdff401c360 comp rx=0 tx=0).stop 2024-03-16T20:45:09.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fe004e30640 1 --2- 172.21.15.122:0/892447008 >> v2:172.21.15.122:3300/0 conn(0x7fe0000b5190 0x7fe0001435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fe004e30640 1 -- 172.21.15.122:0/892447008 >> 172.21.15.122:0/892447008 conn(0x7fe00009de70 msgr2=0x7fe0000b86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:09.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f0295125640 1 -- 172.21.15.122:0/1171526984 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02800ab600 con 0x7f029014ef60 2024-03-16T20:45:09.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f0295125640 1 -- 172.21.15.122:0/1171526984 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02800a7050 con 0x7f029014ef60 2024-03-16T20:45:09.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f0295125640 1 -- 172.21.15.122:0/1171526984 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02800ab980 con 0x7f029014ef60 2024-03-16T20:45:09.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f1e4be27640 1 -- start start 2024-03-16T20:45:09.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f23ceffd640 1 --2- 172.21.15.122:0/640065492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23d0095180 0x7f23d00955a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f23c4008800 tx=0x7f23c401e9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cea14da5455f4522 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f99ae1e5640 1 -- 172.21.15.122:0/770670872 shutdown_connections 2024-03-16T20:45:09.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f029838b640 1 -- 172.21.15.122:0/1171526984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f029014ef60 msgr2=0x7f029014f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f029838b640 1 --2- 172.21.15.122:0/1171526984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f029014ef60 0x7f029014f3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0280009870 tx=0x7f028001c490 comp rx=0 tx=0).stop 2024-03-16T20:45:09.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f23ce7fc640 1 -- 172.21.15.122:0/640065492 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c40a7880 con 0x7f23d0095180 2024-03-16T20:45:09.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f029838b640 1 -- 172.21.15.122:0/1171526984 shutdown_connections 2024-03-16T20:45:09.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f029838b640 1 --2- 172.21.15.122:0/1171526984 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f029014f900 0x7f02901563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f029838b640 1 --2- 172.21.15.122:0/1171526984 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f029014ef60 0x7f029014f3c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f029838b640 1 -- 172.21.15.122:0/1171526984 >> 172.21.15.122:0/1171526984 conn(0x7f02900b8d10 msgr2=0x7f029009a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:09.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f23ce7fc640 1 -- 172.21.15.122:0/640065492 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c40ab070 con 0x7f23d0095180 2024-03-16T20:45:09.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f029838b640 1 -- 172.21.15.122:0/1171526984 shutdown_connections 2024-03-16T20:45:09.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f029838b640 1 -- 172.21.15.122:0/1171526984 wait complete. 2024-03-16T20:45:09.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fab7e7fc640 1 -- 172.21.15.122:0/1549356665 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab700ab650 con 0x7fab800b71d0 2024-03-16T20:45:09.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f23ce7fc640 1 -- 172.21.15.122:0/640065492 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c40a7c00 con 0x7f23d0095180 2024-03-16T20:45:09.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fab84c74640 1 -- 172.21.15.122:0/1549356665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab800b71d0 msgr2=0x7fab800b46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fab84c74640 1 --2- 172.21.15.122:0/1549356665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab800b71d0 0x7fab800b46a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab800b92d0 tx=0x7fab7001c450 comp rx=0 tx=0).stop 2024-03-16T20:45:09.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7ff81aa44640 1 -- 172.21.15.122:0/237079349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff80c033920 msgr2=0x7ff80c036540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7ff81aa44640 1 --2- 172.21.15.122:0/237079349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff80c033920 0x7ff80c036540 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff814069370 tx=0x7ff814099660 comp rx=0 tx=0).stop 2024-03-16T20:45:09.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7ff81aa44640 1 -- 172.21.15.122:0/237079349 shutdown_connections 2024-03-16T20:45:09.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7ff81aa44640 1 --2- 172.21.15.122:0/237079349 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff80c033d00 0x7ff80c035560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7ff81aa44640 1 --2- 172.21.15.122:0/237079349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff80c033920 0x7ff80c036540 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff814069370 tx=0x7ff814099660 comp rx=0 tx=0).stop 2024-03-16T20:45:09.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7ff81aa44640 1 -- 172.21.15.122:0/237079349 >> 172.21.15.122:0/237079349 conn(0x7ff80c03a860 msgr2=0x7ff80c03ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:09.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fab84c74640 1 -- 172.21.15.122:0/1549356665 shutdown_connections 2024-03-16T20:45:09.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fab84c74640 1 --2- 172.21.15.122:0/1549356665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab800b71d0 0x7fab800b46a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fab84c74640 1 -- 172.21.15.122:0/1549356665 >> 172.21.15.122:0/1549356665 conn(0x7fab8006cd70 msgr2=0x7fab8006d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:09.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fab84c74640 1 -- 172.21.15.122:0/1549356665 shutdown_connections 2024-03-16T20:45:09.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fe004e30640 1 -- 172.21.15.122:0/892447008 shutdown_connections 2024-03-16T20:45:09.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fe004e30640 1 -- 172.21.15.122:0/892447008 wait complete. 2024-03-16T20:45:09.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7fab84c74640 1 -- 172.21.15.122:0/1549356665 wait complete. 2024-03-16T20:45:09.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7ff81aa44640 1 -- 172.21.15.122:0/237079349 shutdown_connections 2024-03-16T20:45:09.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f23d5fe6640 1 -- 172.21.15.122:0/640065492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23d0095180 msgr2=0x7f23d00955a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f23d5fe6640 1 --2- 172.21.15.122:0/640065492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23d0095180 0x7f23d00955a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f23c4008800 tx=0x7f23c401e9d0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f99ae1e5640 1 -- 172.21.15.122:0/770670872 wait complete. 2024-03-16T20:45:09.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7f23d5fe6640 1 -- 172.21.15.122:0/640065492 shutdown_connections 2024-03-16T20:45:09.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f23d5fe6640 1 --2- 172.21.15.122:0/640065492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23d0095180 0x7f23d00955a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f23d5fe6640 1 -- 172.21.15.122:0/640065492 >> 172.21.15.122:0/640065492 conn(0x7f23d009a4c0 msgr2=0x7f23d009a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:09.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f029838b640 1 Processor -- start 2024-03-16T20:45:09.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f23d5fe6640 1 -- 172.21.15.122:0/640065492 shutdown_connections 2024-03-16T20:45:09.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f23d5fe6640 1 -- 172.21.15.122:0/640065492 wait complete. 2024-03-16T20:45:09.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f23d5fe6640 1 Processor -- start 2024-03-16T20:45:09.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab84c74640 1 Processor -- start 2024-03-16T20:45:09.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab84c74640 1 -- start start 2024-03-16T20:45:09.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.916+0000 7ff81aa44640 1 -- 172.21.15.122:0/237079349 wait complete. 2024-03-16T20:45:09.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7ff81aa44640 1 Processor -- start 2024-03-16T20:45:09.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99ae1e5640 1 Processor -- start 2024-03-16T20:45:09.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99ae1e5640 1 -- start start 2024-03-16T20:45:09.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99ae1e5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99a0033950 0x7f99a002c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99ae1e5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99a0033d30 0x7f99a0172190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99ae1e5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99a00c8630 0x7f99a01726d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99ae1e5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a0171d40 con 0x7f99a0033d30 2024-03-16T20:45:09.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99ae1e5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a002d540 con 0x7f99a00c8630 2024-03-16T20:45:09.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99ae1e5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a00c5ee0 con 0x7f99a0033950 2024-03-16T20:45:09.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f1e49bc3640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f1e440b48e0 0x7f1e44143d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.122:51246/0 (socket says 172.21.15.122:51246) 2024-03-16T20:45:09.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fe004e30640 1 Processor -- start 2024-03-16T20:45:09.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f029838b640 1 -- start start 2024-03-16T20:45:09.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f029838b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f029009ae30 0x7f0290071f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f029838b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f029014ef60 0x7f029008bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f029838b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f029014f900 0x7f029008c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f029838b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029008cb90 con 0x7f029014ef60 2024-03-16T20:45:09.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f029838b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029008e6e0 con 0x7f029014f900 2024-03-16T20:45:09.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f029838b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f029008d970 con 0x7f029009ae30 2024-03-16T20:45:09.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f0296928640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f029014f900 0x7f029008c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f0296928640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f029014f900 0x7f029008c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.122:45090/0 (socket says 172.21.15.122:45090) 2024-03-16T20:45:09.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f0296928640 1 -- 172.21.15.122:0/3638449101 learned_addr learned my addr 172.21.15.122:0/3638449101 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:09.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f0295926640 1 --2- 172.21.15.122:0/3638449101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f029014ef60 0x7f029008bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f0296127640 1 --2- 172.21.15.122:0/3638449101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f029009ae30 0x7f0290071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.945 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f0296928640 1 -- 172.21.15.122:0/3638449101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f029009ae30 msgr2=0x7f0290071f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f0296928640 1 --2- 172.21.15.122:0/3638449101 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f029009ae30 0x7f0290071f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f0296928640 1 -- 172.21.15.122:0/3638449101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f029014ef60 msgr2=0x7f029008bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f0296928640 1 --2- 172.21.15.122:0/3638449101 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f029014ef60 0x7f029008bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f0296928640 1 -- 172.21.15.122:0/3638449101 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02800094d0 con 0x7f029014f900 2024-03-16T20:45:09.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab84c74640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab800b4be0 0x7fab80070b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99a7fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99a00c8630 0x7f99a01726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99a7fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99a00c8630 0x7f99a01726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.122:45094/0 (socket says 172.21.15.122:45094) 2024-03-16T20:45:09.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99a7fff640 1 -- 172.21.15.122:0/3792156773 learned_addr learned my addr 172.21.15.122:0/3792156773 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:09.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99a6ffd640 1 --2- 172.21.15.122:0/3792156773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99a0033d30 0x7f99a0172190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab84c74640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab800b71d0 0x7fab80149fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7ff81aa44640 1 -- start start 2024-03-16T20:45:09.947 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab84c74640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab800b8d00 0x7fab8014a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99a77fe640 1 --2- 172.21.15.122:0/3792156773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99a0033950 0x7f99a002c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab7f7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab800b4be0 0x7fab80070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.948 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fe004e30640 1 -- start start 2024-03-16T20:45:09.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab7f7fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab800b4be0 0x7fab80070b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.122:45104/0 (socket says 172.21.15.122:45104) 2024-03-16T20:45:09.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab7f7fe640 1 -- 172.21.15.122:0/2640283853 learned_addr learned my addr 172.21.15.122:0/2640283853 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:09.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fe004e30640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0000b5190 0x7fe000072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fe004e30640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0000b8db0 0x7fe0001e4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fe004e30640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe000154520 0x7fe0001e7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fe004e30640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe000153730 con 0x7fe000154520 2024-03-16T20:45:09.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fe004e30640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe000154fb0 con 0x7fe0000b5190 2024-03-16T20:45:09.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab84c74640 1 -- 172.21.15.122:0/2640283853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab80149b30 con 0x7fab800b8d00 2024-03-16T20:45:09.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fe004e30640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe000153980 con 0x7fe0000b8db0 2024-03-16T20:45:09.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab84c74640 1 -- 172.21.15.122:0/2640283853 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab800717d0 con 0x7fab800b4be0 2024-03-16T20:45:09.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f1e49bc3640 1 -- 172.21.15.122:0/3133210254 learned_addr learned my addr 172.21.15.122:0/3133210254 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:09.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab84c74640 1 -- 172.21.15.122:0/2640283853 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab80072540 con 0x7fab800b71d0 2024-03-16T20:45:09.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99a7fff640 1 -- 172.21.15.122:0/3792156773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99a0033950 msgr2=0x7f99a002c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99a7fff640 1 --2- 172.21.15.122:0/3792156773 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f99a0033950 0x7f99a002c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99a7fff640 1 -- 172.21.15.122:0/3792156773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99a0033d30 msgr2=0x7f99a0172190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99a7fff640 1 --2- 172.21.15.122:0/3792156773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99a0033d30 0x7f99a0172190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99a7fff640 1 -- 172.21.15.122:0/3792156773 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9994008550 con 0x7f99a00c8630 2024-03-16T20:45:09.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab7ffff640 1 --2- 172.21.15.122:0/2640283853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab800b8d00 0x7fab8014a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fdffdd9b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0000b8db0 0x7fe0001e4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fdffed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe000154520 0x7fe0001e7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fdffed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe000154520 0x7fe0001e7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41282/0 (socket says 172.21.15.122:41282) 2024-03-16T20:45:09.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fdffed9d640 1 -- 172.21.15.122:0/3147738962 learned_addr learned my addr 172.21.15.122:0/3147738962 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:09.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f0296928640 1 --2- 172.21.15.122:0/3638449101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f029014f900 0x7f029008c100 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f028800cc30 tx=0x7f028800a5c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fdffe59c640 1 --2- 172.21.15.122:0/3147738962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0000b5190 0x7fe000072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab7effd640 1 --2- 172.21.15.122:0/2640283853 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab800b71d0 0x7fab80149fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99a7fff640 1 --2- 172.21.15.122:0/3792156773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99a00c8630 0x7f99a01726d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99a8068660 tx=0x7f99a8068a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fdffed9d640 1 -- 172.21.15.122:0/3147738962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0000b8db0 msgr2=0x7fe0001e4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fdffed9d640 1 --2- 172.21.15.122:0/3147738962 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe0000b8db0 0x7fe0001e4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fdffed9d640 1 -- 172.21.15.122:0/3147738962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0000b5190 msgr2=0x7fe000072aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fdffed9d640 1 --2- 172.21.15.122:0/3147738962 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe0000b5190 0x7fe000072aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fdffed9d640 1 -- 172.21.15.122:0/3147738962 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff4009420 con 0x7fe000154520 2024-03-16T20:45:09.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab7ffff640 1 -- 172.21.15.122:0/2640283853 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab800b71d0 msgr2=0x7fab80149fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab7ffff640 1 --2- 172.21.15.122:0/2640283853 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fab800b71d0 0x7fab80149fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab7ffff640 1 -- 172.21.15.122:0/2640283853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab800b4be0 msgr2=0x7fab80070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab7ffff640 1 --2- 172.21.15.122:0/2640283853 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fab800b4be0 0x7fab80070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fab7ffff640 1 -- 172.21.15.122:0/2640283853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab70009420 con 0x7fab800b8d00 2024-03-16T20:45:09.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7ff81aa44640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff80c033d00 0x7ff80c0c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7ff81aa44640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff80c0c8600 0x7ff80c16cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7ff81aa44640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff80c0b8fd0 0x7ff80c0b9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7ff81aa44640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c16d890 con 0x7ff80c0c8600 2024-03-16T20:45:09.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7ff81aa44640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c0c5ea0 con 0x7ff80c033d00 2024-03-16T20:45:09.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7ff81aa44640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c16d110 con 0x7ff80c0b8fd0 2024-03-16T20:45:09.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f1e4be27640 1 --2- 172.21.15.122:0/3133210254 >> v2:172.21.15.122:3300/0 conn(0x7f1e440adad0 0x7f1e44140d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f1e4be27640 1 --2- 172.21.15.122:0/3133210254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e44141270 0x7f1e4414e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f1e4be27640 1 -- 172.21.15.122:0/3133210254 --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e440b4cc0 con 0x7f1e440b48e0 2024-03-16T20:45:09.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f1e4be27640 1 -- 172.21.15.122:0/3133210254 --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e441481a0 con 0x7f1e440adad0 2024-03-16T20:45:09.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f1e4be27640 1 -- 172.21.15.122:0/3133210254 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440702a0 con 0x7f1e44141270 2024-03-16T20:45:09.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fdffed9d640 1 --2- 172.21.15.122:0/3147738962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe000154520 0x7fe0001e7790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdff800c730 tx=0x7fdff800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f0286ffd640 1 -- 172.21.15.122:0/3638449101 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028800ce40 con 0x7f029014f900 2024-03-16T20:45:09.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f029838b640 1 -- 172.21.15.122:0/3638449101 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029008dba0 con 0x7f029014f900 2024-03-16T20:45:09.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f029838b640 1 -- 172.21.15.122:0/3638449101 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029008e070 con 0x7f029014f900 2024-03-16T20:45:09.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fdfeeffd640 1 -- 172.21.15.122:0/3147738962 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8003b80 con 0x7fe000154520 2024-03-16T20:45:09.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f9993fff640 1 -- 172.21.15.122:0/3792156773 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a808f070 con 0x7f99a00c8630 2024-03-16T20:45:09.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f9993fff640 1 -- 172.21.15.122:0/3792156773 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99a80683d0 con 0x7f99a00c8630 2024-03-16T20:45:09.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f9993fff640 1 -- 172.21.15.122:0/3792156773 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99a8074c00 con 0x7f99a00c8630 2024-03-16T20:45:09.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fe004e30640 1 -- 172.21.15.122:0/3147738962 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0000adbf0 con 0x7fe000154520 2024-03-16T20:45:09.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99ae1e5640 1 -- 172.21.15.122:0/3792156773 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a0037440 con 0x7f99a00c8630 2024-03-16T20:45:09.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7ff813fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff80c033d00 0x7ff80c0c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7ff813fff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff80c033d00 0x7ff80c0c2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.122:45112/0 (socket says 172.21.15.122:45112) 2024-03-16T20:45:09.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7ff813fff640 1 -- 172.21.15.122:0/282025606 learned_addr learned my addr 172.21.15.122:0/282025606 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:09.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7fe004e30640 1 -- 172.21.15.122:0/3147738962 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00014b580 con 0x7fe000154520 2024-03-16T20:45:09.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7ff8137fe640 1 --2- 172.21.15.122:0/282025606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff80c0c8600 0x7ff80c16cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7ff818fe1640 1 --2- 172.21.15.122:0/282025606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff80c0b8fd0 0x7ff80c0b9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f0286ffd640 1 -- 172.21.15.122:0/3638449101 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02880039a0 con 0x7f029014f900 2024-03-16T20:45:09.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f0286ffd640 1 -- 172.21.15.122:0/3638449101 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0288003d30 con 0x7f029014f900 2024-03-16T20:45:09.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f23d5fe6640 1 -- start start 2024-03-16T20:45:09.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f23d5fe6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23d0095180 0x7f23d0155e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f23d5fe6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23d00b4a40 0x7f23d0097870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f23d5fe6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23d0153ea0 0x7f23d02009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f23d5fe6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d0099fc0 con 0x7f23d0095180 2024-03-16T20:45:09.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f23d5fe6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d0156390 con 0x7f23d0153ea0 2024-03-16T20:45:09.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f23d5fe6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d0099900 con 0x7f23d00b4a40 2024-03-16T20:45:09.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f23cffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23d0153ea0 0x7f23d02009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f23cffff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23d0153ea0 0x7f23d02009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.122:45120/0 (socket says 172.21.15.122:45120) 2024-03-16T20:45:09.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f23cffff640 1 -- 172.21.15.122:0/1664193591 learned_addr learned my addr 172.21.15.122:0/1664193591 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:09.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7ff813fff640 1 -- 172.21.15.122:0/282025606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff80c0b8fd0 msgr2=0x7ff80c0b9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7ff813fff640 1 --2- 172.21.15.122:0/282025606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff80c0b8fd0 0x7ff80c0b9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7ff813fff640 1 -- 172.21.15.122:0/282025606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff80c0c8600 msgr2=0x7ff80c16cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7ff813fff640 1 --2- 172.21.15.122:0/282025606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff80c0c8600 0x7ff80c16cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7ff813fff640 1 -- 172.21.15.122:0/282025606 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80c0c4440 con 0x7ff80c033d00 2024-03-16T20:45:09.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7ff818fe1640 1 --2- 172.21.15.122:0/282025606 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff80c0b8fd0 0x7ff80c0b9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:09.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f23ceffd640 1 --2- 172.21.15.122:0/1664193591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23d00b4a40 0x7f23d0097870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.920+0000 7f99ae1e5640 1 -- 172.21.15.122:0/3792156773 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a00c5c00 con 0x7f99a00c8630 2024-03-16T20:45:09.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f1e48bc1640 1 -- 172.21.15.122:0/3133210254 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2014426435 0 0) 0x7f1e440b4cc0 con 0x7f1e440b48e0 2024-03-16T20:45:09.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f1e48bc1640 1 -- 172.21.15.122:0/3133210254 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e30003530 con 0x7f1e440b48e0 2024-03-16T20:45:09.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f23ceffd640 1 -- 172.21.15.122:0/1664193591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23d0153ea0 msgr2=0x7f23d02009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f23ceffd640 1 --2- 172.21.15.122:0/1664193591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23d0153ea0 0x7f23d02009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f23ceffd640 1 -- 172.21.15.122:0/1664193591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23d0095180 msgr2=0x7f23d0155e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f23ceffd640 1 --2- 172.21.15.122:0/1664193591 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f23d0095180 0x7f23d0155e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f23ceffd640 1 -- 172.21.15.122:0/1664193591 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c4008430 con 0x7f23d00b4a40 2024-03-16T20:45:09.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f23cffff640 1 --2- 172.21.15.122:0/1664193591 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f23d0153ea0 0x7f23d02009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:45:09.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f23ceffd640 1 --2- 172.21.15.122:0/1664193591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23d00b4a40 0x7f23d0097870 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f23c40af000 tx=0x7f23c401d210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7fab7ffff640 1 --2- 172.21.15.122:0/2640283853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab800b8d00 0x7fab8014a500 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab7400c410 tx=0x7fab7400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f1e4a3c4640 1 --2- 172.21.15.122:0/3133210254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e44141270 0x7f1e4414e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f1e493c2640 1 --2- 172.21.15.122:0/3133210254 >> v2:172.21.15.122:3300/0 conn(0x7f1e440adad0 0x7f1e44140d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7fdfeeffd640 1 -- 172.21.15.122:0/3147738962 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff8003d00 con 0x7fe000154520 2024-03-16T20:45:09.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7fdfeeffd640 1 -- 172.21.15.122:0/3147738962 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff800aa90 con 0x7fe000154520 2024-03-16T20:45:09.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f029838b640 1 -- 172.21.15.122:0/3638449101 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0278001040 con 0x7f029014f900 2024-03-16T20:45:09.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7fe004e30640 1 -- 172.21.15.122:0/3147738962 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdfe4001040 con 0x7fe000154520 2024-03-16T20:45:09.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f23b7fff640 1 -- 172.21.15.122:0/1664193591 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4005ba0 con 0x7f23d00b4a40 2024-03-16T20:45:09.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f23b7fff640 1 -- 172.21.15.122:0/1664193591 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d0157820 con 0x7f23d00b4a40 2024-03-16T20:45:09.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f23d5fe6640 1 -- 172.21.15.122:0/1664193591 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d0099be0 con 0x7f23d00b4a40 2024-03-16T20:45:09.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f23d5fe6640 1 -- 172.21.15.122:0/1664193591 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d0099490 con 0x7f23d00b4a40 2024-03-16T20:45:09.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7fab67fff640 1 -- 172.21.15.122:0/2640283853 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab74003600 con 0x7fab800b8d00 2024-03-16T20:45:09.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7fab67fff640 1 -- 172.21.15.122:0/2640283853 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab74003760 con 0x7fab800b8d00 2024-03-16T20:45:09.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7fab84c74640 1 -- 172.21.15.122:0/2640283853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab80071210 con 0x7fab800b8d00 2024-03-16T20:45:09.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7fab84c74640 1 -- 172.21.15.122:0/2640283853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab8014b7c0 con 0x7fab800b8d00 2024-03-16T20:45:09.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7fab67fff640 1 -- 172.21.15.122:0/2640283853 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab74003b50 con 0x7fab800b8d00 2024-03-16T20:45:09.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f23b7fff640 1 -- 172.21.15.122:0/1664193591 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c4006030 con 0x7f23d00b4a40 2024-03-16T20:45:09.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f99ae1e5640 1 -- 172.21.15.122:0/3792156773 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9988001040 con 0x7f99a00c8630 2024-03-16T20:45:09.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7ff813fff640 1 --2- 172.21.15.122:0/282025606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff80c033d00 0x7ff80c0c2f70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff814069ec0 tx=0x7ff814096450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7ff810ff9640 1 -- 172.21.15.122:0/282025606 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814136610 con 0x7ff80c033d00 2024-03-16T20:45:09.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7ff81aa44640 1 -- 172.21.15.122:0/282025606 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff80c0c5bc0 con 0x7ff80c033d00 2024-03-16T20:45:09.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7ff81aa44640 1 -- 172.21.15.122:0/282025606 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff80c0c4160 con 0x7ff80c033d00 2024-03-16T20:45:09.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7fdfeeffd640 1 -- 172.21.15.122:0/3147738962 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fdff800abf0 con 0x7fe000154520 2024-03-16T20:45:09.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f9993fff640 1 -- 172.21.15.122:0/3792156773 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f99a8094420 con 0x7f99a00c8630 2024-03-16T20:45:09.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7ff810ff9640 1 -- 172.21.15.122:0/282025606 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8140722f0 con 0x7ff80c033d00 2024-03-16T20:45:09.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7ff81aa44640 1 -- 172.21.15.122:0/282025606 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff80c0d5ec0 con 0x7ff80c033d00 2024-03-16T20:45:09.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f9993fff640 1 --2- 172.21.15.122:0/3792156773 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f998c066ec0 0x7f998c069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7ff810ff9640 1 -- 172.21.15.122:0/282025606 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814136970 con 0x7ff80c033d00 2024-03-16T20:45:09.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7fdfeeffd640 1 --2- 172.21.15.122:0/3147738962 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fdfe8067370 0x7fdfe8069830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f9993fff640 1 -- 172.21.15.122:0/3792156773 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99a806f030 con 0x7f99a00c8630 2024-03-16T20:45:09.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f1e4a3c4640 1 -- 172.21.15.122:0/3133210254 >> v2:172.21.15.122:3300/0 conn(0x7f1e440adad0 msgr2=0x7f1e44140d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:09.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f1e4a3c4640 1 --2- 172.21.15.122:0/3133210254 >> v2:172.21.15.122:3300/0 conn(0x7f1e440adad0 0x7f1e44140d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f1e4a3c4640 1 -- 172.21.15.122:0/3133210254 >> v1:172.21.15.195:6789/0 conn(0x7f1e440b48e0 legacy=0x7f1e44143d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f1e4a3c4640 1 -- 172.21.15.122:0/3133210254 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e441476c0 con 0x7f1e44141270 2024-03-16T20:45:09.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f0286ffd640 1 -- 172.21.15.122:0/3638449101 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f0288003e90 con 0x7f029014f900 2024-03-16T20:45:09.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f0286ffd640 1 --2- 172.21.15.122:0/3638449101 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0274066e40 0x7f0274069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f0286ffd640 1 -- 172.21.15.122:0/3638449101 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f02880852f0 con 0x7f029014f900 2024-03-16T20:45:09.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f0286ffd640 1 --2- 172.21.15.122:0/3638449101 >> [v2:172.21.15.195:6824/3035467983,v1:172.21.15.195:6825/3035467983] conn(0x7f027406f0c0 0x7f02740714e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f0286ffd640 1 -- 172.21.15.122:0/3638449101 --> [v2:172.21.15.195:6824/3035467983,v1:172.21.15.195:6825/3035467983] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0274071b90 con 0x7f027406f0c0 2024-03-16T20:45:09.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f0286ffd640 1 -- 172.21.15.122:0/3638449101 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02880856a0 con 0x7f029014f900 2024-03-16T20:45:09.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.924+0000 7f23d5fe6640 1 -- 172.21.15.122:0/1664193591 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f23d0099740 con 0x7f23d00b4a40 2024-03-16T20:45:09.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fdfeeffd640 1 -- 172.21.15.122:0/3147738962 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdff8085230 con 0x7fe000154520 2024-03-16T20:45:09.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fdffe59c640 1 --2- 172.21.15.122:0/3147738962 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fdfe8067370 0x7fdfe8069830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f0296127640 1 --2- 172.21.15.122:0/3638449101 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0274066e40 0x7f0274069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f9993fff640 1 --2- 172.21.15.122:0/3792156773 >> [v2:172.21.15.122:6816/362583029,v1:172.21.15.122:6817/362583029] conn(0x7f998c06f060 0x7f998c071480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f9993fff640 1 -- 172.21.15.122:0/3792156773 --> [v2:172.21.15.122:6816/362583029,v1:172.21.15.122:6817/362583029] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f998c071b30 con 0x7f998c06f060 2024-03-16T20:45:09.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f99a6ffd640 1 --2- 172.21.15.122:0/3792156773 >> [v2:172.21.15.122:6816/362583029,v1:172.21.15.122:6817/362583029] conn(0x7f998c06f060 0x7f998c071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fdfeeffd640 1 --2- 172.21.15.122:0/3147738962 >> [v2:172.21.15.195:6810/1730924554,v1:172.21.15.195:6811/1730924554] conn(0x7fdfe806f5f0 0x7fdfe8071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f0295926640 1 --2- 172.21.15.122:0/3638449101 >> [v2:172.21.15.195:6824/3035467983,v1:172.21.15.195:6825/3035467983] conn(0x7f027406f0c0 0x7f02740714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fab67fff640 1 -- 172.21.15.122:0/2640283853 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fab74003cb0 con 0x7fab800b8d00 2024-03-16T20:45:09.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fab84c74640 1 -- 172.21.15.122:0/2640283853 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab60001040 con 0x7fab800b8d00 2024-03-16T20:45:09.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fdfeeffd640 1 -- 172.21.15.122:0/3147738962 --> [v2:172.21.15.195:6810/1730924554,v1:172.21.15.195:6811/1730924554] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdfe80720c0 con 0x7fdfe806f5f0 2024-03-16T20:45:09.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fdfeeffd640 1 -- 172.21.15.122:0/3147738962 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdff80855e0 con 0x7fe000154520 2024-03-16T20:45:09.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f99a77fe640 1 --2- 172.21.15.122:0/3792156773 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f998c066ec0 0x7f998c069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f1e4a3c4640 1 --2- 172.21.15.122:0/3133210254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e44141270 0x7f1e4414e610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1e340097c0 tx=0x7f1e3401e610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d7047f0f7478bb3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fdffdd9b640 1 --2- 172.21.15.122:0/3147738962 >> [v2:172.21.15.195:6810/1730924554,v1:172.21.15.195:6811/1730924554] conn(0x7fdfe806f5f0 0x7fdfe8071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fab67fff640 1 --2- 172.21.15.122:0/2640283853 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fab6c066e90 0x7fab6c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f1e48bc1640 1 -- 172.21.15.122:0/3133210254 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e340ac680 con 0x7f1e44141270 2024-03-16T20:45:09.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f9993fff640 1 -- 172.21.15.122:0/3792156773 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99a80cc790 con 0x7f99a00c8630 2024-03-16T20:45:09.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f0296127640 1 --2- 172.21.15.122:0/3638449101 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0274066e40 0x7f0274069300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f028c001600 tx=0x7f028c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f99a6ffd640 1 --2- 172.21.15.122:0/3792156773 >> [v2:172.21.15.122:6816/362583029,v1:172.21.15.122:6817/362583029] conn(0x7f998c06f060 0x7f998c071480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f99a77fe640 1 --2- 172.21.15.122:0/3792156773 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f998c066ec0 0x7f998c069380 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f99940b2000 tx=0x7f999400a040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f0295926640 1 --2- 172.21.15.122:0/3638449101 >> [v2:172.21.15.195:6824/3035467983,v1:172.21.15.195:6825/3035467983] conn(0x7f027406f0c0 0x7f02740714e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fab67fff640 1 -- 172.21.15.122:0/2640283853 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab74083e20 con 0x7fab800b8d00 2024-03-16T20:45:09.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fab67fff640 1 --2- 172.21.15.122:0/2640283853 >> [v2:172.21.15.122:6800/685793600,v1:172.21.15.122:6801/685793600] conn(0x7fab6c06f0b0 0x7fab6c0714d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fab67fff640 1 -- 172.21.15.122:0/2640283853 --> [v2:172.21.15.122:6800/685793600,v1:172.21.15.122:6801/685793600] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab6c071b80 con 0x7fab6c06f0b0 2024-03-16T20:45:09.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fab67fff640 1 -- 172.21.15.122:0/2640283853 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab74086530 con 0x7fab800b8d00 2024-03-16T20:45:09.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fab7f7fe640 1 --2- 172.21.15.122:0/2640283853 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fab6c066e90 0x7fab6c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.975 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fdffe59c640 1 --2- 172.21.15.122:0/3147738962 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fdfe8067370 0x7fdfe8069830 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe000153240 tx=0x7fdff0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f23b7fff640 1 -- 172.21.15.122:0/1664193591 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f23c4006190 con 0x7f23d00b4a40 2024-03-16T20:45:09.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fab7effd640 1 --2- 172.21.15.122:0/2640283853 >> [v2:172.21.15.122:6800/685793600,v1:172.21.15.122:6801/685793600] conn(0x7fab6c06f0b0 0x7fab6c0714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fab7f7fe640 1 --2- 172.21.15.122:0/2640283853 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fab6c066e90 0x7fab6c069350 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fab78005c10 tx=0x7fab78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fab7effd640 1 --2- 172.21.15.122:0/2640283853 >> [v2:172.21.15.122:6800/685793600,v1:172.21.15.122:6801/685793600] conn(0x7fab6c06f0b0 0x7fab6c0714d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fdffdd9b640 1 --2- 172.21.15.122:0/3147738962 >> [v2:172.21.15.195:6810/1730924554,v1:172.21.15.195:6811/1730924554] conn(0x7fdfe806f5f0 0x7fdfe8071a10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f23b7fff640 1 --2- 172.21.15.122:0/1664193591 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f23bc066e40 0x7f23bc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f1e48bc1640 1 -- 172.21.15.122:0/3133210254 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e340ac7e0 con 0x7f1e44141270 2024-03-16T20:45:09.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f1e48bc1640 1 -- 172.21.15.122:0/3133210254 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e340acb90 con 0x7f1e44141270 2024-03-16T20:45:09.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f23b7fff640 1 -- 172.21.15.122:0/1664193591 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23c411ed00 con 0x7f23d00b4a40 2024-03-16T20:45:09.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f9993fff640 1 -- 172.21.15.122:0/3792156773 <== osd.3 v2:172.21.15.122:6816/362583029 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f998c071b30 con 0x7f998c06f060 2024-03-16T20:45:09.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f23b7fff640 1 --2- 172.21.15.122:0/1664193591 >> [v2:172.21.15.122:6808/4193825355,v1:172.21.15.122:6809/4193825355] conn(0x7f23bc06efe0 0x7f23bc071400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f23b7fff640 1 -- 172.21.15.122:0/1664193591 --> [v2:172.21.15.122:6808/4193825355,v1:172.21.15.122:6809/4193825355] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f23bc071ab0 con 0x7f23bc06efe0 2024-03-16T20:45:09.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f23b7fff640 1 -- 172.21.15.122:0/1664193591 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23c411f150 con 0x7f23d00b4a40 2024-03-16T20:45:09.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f23cffff640 1 --2- 172.21.15.122:0/1664193591 >> [v2:172.21.15.122:6808/4193825355,v1:172.21.15.122:6809/4193825355] conn(0x7f23bc06efe0 0x7f23bc071400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f23cf7fe640 1 --2- 172.21.15.122:0/1664193591 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f23bc066e40 0x7f23bc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f1e4be27640 1 -- 172.21.15.122:0/3133210254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e44141270 msgr2=0x7f1e4414e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f1e4be27640 1 --2- 172.21.15.122:0/3133210254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e44141270 0x7f1e4414e610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1e340097c0 tx=0x7f1e3401e610 comp rx=0 tx=0).stop 2024-03-16T20:45:09.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f1e4be27640 1 -- 172.21.15.122:0/3133210254 shutdown_connections 2024-03-16T20:45:09.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f1e4be27640 1 --2- 172.21.15.122:0/3133210254 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e44141270 0x7f1e4414e610 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f1e4be27640 1 --2- 172.21.15.122:0/3133210254 >> v2:172.21.15.122:3300/0 conn(0x7f1e440adad0 0x7f1e44140d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f1e4be27640 1 -- 172.21.15.122:0/3133210254 >> 172.21.15.122:0/3133210254 conn(0x7f1e440b96b0 msgr2=0x7f1e4406af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:09.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f1e4be27640 1 -- 172.21.15.122:0/3133210254 shutdown_connections 2024-03-16T20:45:09.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f1e4be27640 1 -- 172.21.15.122:0/3133210254 wait complete. 2024-03-16T20:45:09.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f0286ffd640 1 -- 172.21.15.122:0/3638449101 <== osd.7 v2:172.21.15.195:6824/3035467983 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0274071b90 con 0x7f027406f0c0 2024-03-16T20:45:09.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7f23cffff640 1 --2- 172.21.15.122:0/1664193591 >> [v2:172.21.15.122:6808/4193825355,v1:172.21.15.122:6809/4193825355] conn(0x7f23bc06efe0 0x7f23bc071400 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7fab67fff640 1 -- 172.21.15.122:0/2640283853 <== osd.1 v2:172.21.15.122:6800/685793600 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fab6c071b80 con 0x7fab6c06f0b0 2024-03-16T20:45:09.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7fdfeeffd640 1 -- 172.21.15.122:0/3147738962 <== osd.6 v2:172.21.15.195:6810/1730924554 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fdfe80720c0 con 0x7fdfe806f5f0 2024-03-16T20:45:09.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f23cf7fe640 1 --2- 172.21.15.122:0/1664193591 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f23bc066e40 0x7f23bc069300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f23c0000c10 tx=0x7f23c0007910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7ff810ff9640 1 -- 172.21.15.122:0/282025606 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7ff814072040 con 0x7ff80c033d00 2024-03-16T20:45:09.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7ff810ff9640 1 --2- 172.21.15.122:0/282025606 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff7f8066e70 0x7ff7f8069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.928+0000 7ff810ff9640 1 -- 172.21.15.122:0/282025606 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff81406f040 con 0x7ff80c033d00 2024-03-16T20:45:09.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7ff810ff9640 1 --2- 172.21.15.122:0/282025606 >> [v2:172.21.15.122:6819/2377812328,v1:172.21.15.122:6822/2377812328] conn(0x7ff7f806f070 0x7ff7f8071490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7ff810ff9640 1 -- 172.21.15.122:0/282025606 --> [v2:172.21.15.122:6819/2377812328,v1:172.21.15.122:6822/2377812328] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff7f8071b40 con 0x7ff7f806f070 2024-03-16T20:45:09.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7ff810ff9640 1 -- 172.21.15.122:0/282025606 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff814199da0 con 0x7ff80c033d00 2024-03-16T20:45:09.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7ff8137fe640 1 --2- 172.21.15.122:0/282025606 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff7f8066e70 0x7ff7f8069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e4be27640 1 Processor -- start 2024-03-16T20:45:09.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f23b7fff640 1 -- 172.21.15.122:0/1664193591 <== osd.0 v2:172.21.15.122:6808/4193825355 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f23bc071ab0 con 0x7f23bc06efe0 2024-03-16T20:45:09.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7ff8137fe640 1 --2- 172.21.15.122:0/282025606 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff7f8066e70 0x7ff7f8069330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff80c005790 tx=0x7ff800008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e4be27640 1 -- start start 2024-03-16T20:45:09.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e4be27640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e440adad0 0x7f1e44146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e4be27640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e440b48e0 0x7f1e441e7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e4be27640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e44141270 0x7f1e441e7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e4be27640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e441e7b90 con 0x7f1e44141270 2024-03-16T20:45:09.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e4be27640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4413c8a0 con 0x7f1e440adad0 2024-03-16T20:45:09.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e4be27640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440afff0 con 0x7f1e440b48e0 2024-03-16T20:45:09.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7ff818fe1640 1 --2- 172.21.15.122:0/282025606 >> [v2:172.21.15.122:6819/2377812328,v1:172.21.15.122:6822/2377812328] conn(0x7ff7f806f070 0x7ff7f8071490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e49bc3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e440adad0 0x7f1e44146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e49bc3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e440adad0 0x7f1e44146390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.122:45126/0 (socket says 172.21.15.122:45126) 2024-03-16T20:45:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e49bc3640 1 -- 172.21.15.122:0/2554488820 learned_addr learned my addr 172.21.15.122:0/2554488820 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:09.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7ff818fe1640 1 --2- 172.21.15.122:0/282025606 >> [v2:172.21.15.122:6819/2377812328,v1:172.21.15.122:6822/2377812328] conn(0x7ff7f806f070 0x7ff7f8071490 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e493c2640 1 --2- 172.21.15.122:0/2554488820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e440b48e0 0x7f1e441e7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e49bc3640 1 -- 172.21.15.122:0/2554488820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e440b48e0 msgr2=0x7f1e441e7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e49bc3640 1 --2- 172.21.15.122:0/2554488820 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1e440b48e0 0x7f1e441e7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e49bc3640 1 -- 172.21.15.122:0/2554488820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e44141270 msgr2=0x7f1e441e7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e49bc3640 1 --2- 172.21.15.122:0/2554488820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1e44141270 0x7f1e441e7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e49bc3640 1 -- 172.21.15.122:0/2554488820 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e34009420 con 0x7f1e440adad0 2024-03-16T20:45:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e49bc3640 1 --2- 172.21.15.122:0/2554488820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e440adad0 0x7f1e44146390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00cbf0 tx=0x7f1e3c00a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e3a7fc640 1 -- 172.21.15.122:0/2554488820 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c00ad50 con 0x7f1e440adad0 2024-03-16T20:45:09.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e3a7fc640 1 -- 172.21.15.122:0/2554488820 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3c012400 con 0x7f1e440adad0 2024-03-16T20:45:09.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e4be27640 1 -- 172.21.15.122:0/2554488820 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e440b13b0 con 0x7f1e440adad0 2024-03-16T20:45:09.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e3a7fc640 1 -- 172.21.15.122:0/2554488820 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c012790 con 0x7f1e440adad0 2024-03-16T20:45:09.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.932+0000 7f1e4be27640 1 -- 172.21.15.122:0/2554488820 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e44146bd0 con 0x7f1e440adad0 2024-03-16T20:45:09.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.936+0000 7f1e3a7fc640 1 -- 172.21.15.122:0/2554488820 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f1e3c007070 con 0x7f1e440adad0 2024-03-16T20:45:09.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.936+0000 7f1e3a7fc640 1 --2- 172.21.15.122:0/2554488820 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1e28066e40 0x7f1e28069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.936+0000 7f1e3a7fc640 1 -- 172.21.15.122:0/2554488820 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e3c084a70 con 0x7f1e440adad0 2024-03-16T20:45:09.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.936+0000 7f1e493c2640 1 --2- 172.21.15.122:0/2554488820 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1e28066e40 0x7f1e28069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.936+0000 7f1e493c2640 1 --2- 172.21.15.122:0/2554488820 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1e28066e40 0x7f1e28069300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1e4406df90 tx=0x7f1e40009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7ff810ff9640 1 -- 172.21.15.122:0/282025606 <== osd.2 v2:172.21.15.122:6819/2377812328 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff7f8071b40 con 0x7ff7f806f070 2024-03-16T20:45:09.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f04d80bd640 1 Processor -- start 2024-03-16T20:45:09.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f04d80bd640 1 -- start start 2024-03-16T20:45:09.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f1e4be27640 1 --2- 172.21.15.122:0/2554488820 >> [v2:172.21.15.195:6807/3059274194,v1:172.21.15.195:6809/3059274194] conn(0x7f1e440b0ed0 0x7f1e4413dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f1e4be27640 1 -- 172.21.15.122:0/2554488820 --> [v2:172.21.15.195:6807/3059274194,v1:172.21.15.195:6809/3059274194] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e44141700 con 0x7f1e440b0ed0 2024-03-16T20:45:09.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f1e4a3c4640 1 --2- 172.21.15.122:0/2554488820 >> [v2:172.21.15.195:6807/3059274194,v1:172.21.15.195:6809/3059274194] conn(0x7f1e440b0ed0 0x7f1e4413dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f1e4a3c4640 1 --2- 172.21.15.122:0/2554488820 >> [v2:172.21.15.195:6807/3059274194,v1:172.21.15.195:6809/3059274194] conn(0x7f1e440b0ed0 0x7f1e4413dd30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f04d80bd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f04d0141580 0x7f04d00baac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f04d80bd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04d006d350 0x7f04d0151920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f04d5658640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f04d00bb000 0x7f04d006cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58544/0 (socket says 172.21.15.122:58544) 2024-03-16T20:45:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f04d5658640 1 -- 172.21.15.122:0/2722538775 learned_addr learned my addr 172.21.15.122:0/2722538775 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f04d80bd640 1 -- 172.21.15.122:0/2722538775 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d0147ff0 con 0x7f04d00bb000 2024-03-16T20:45:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f04d80bd640 1 -- 172.21.15.122:0/2722538775 --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d00bb460 con 0x7f04d0141580 2024-03-16T20:45:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f04d80bd640 1 -- 172.21.15.122:0/2722538775 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d01479a0 con 0x7f04d006d350 2024-03-16T20:45:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f04d665a640 1 --2- 172.21.15.122:0/2722538775 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04d006d350 0x7f04d0151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f04d5e59640 1 --2- 172.21.15.122:0/2722538775 >> v2:172.21.15.122:3300/0 conn(0x7f04d0141580 0x7f04d00baac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f04d665a640 1 -- 172.21.15.122:0/2722538775 >> v2:172.21.15.122:3300/0 conn(0x7f04d0141580 msgr2=0x7f04d00baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f04d665a640 1 --2- 172.21.15.122:0/2722538775 >> v2:172.21.15.122:3300/0 conn(0x7f04d0141580 0x7f04d00baac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f04d665a640 1 -- 172.21.15.122:0/2722538775 >> v1:172.21.15.122:6789/0 conn(0x7f04d00bb000 legacy=0x7f04d006cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f04d665a640 1 -- 172.21.15.122:0/2722538775 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d0147c80 con 0x7f04d006d350 2024-03-16T20:45:09.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f04d5e59640 1 --2- 172.21.15.122:0/2722538775 >> v2:172.21.15.122:3300/0 conn(0x7f04d0141580 0x7f04d00baac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:09.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.940+0000 7f1e3a7fc640 1 -- 172.21.15.122:0/2554488820 <== osd.4 v2:172.21.15.195:6807/3059274194 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f1e340ac5a0 con 0x7f1e440b0ed0 2024-03-16T20:45:09.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.944+0000 7f04d665a640 1 --2- 172.21.15.122:0/2722538775 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04d006d350 0x7f04d0151920 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009d90 tx=0x7f04c801c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4bef88678647790 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.944+0000 7f04d4e57640 1 -- 172.21.15.122:0/2722538775 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c80a5940 con 0x7f04d006d350 2024-03-16T20:45:09.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.944+0000 7f04d4e57640 1 -- 172.21.15.122:0/2722538775 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c80a5aa0 con 0x7f04d006d350 2024-03-16T20:45:09.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.944+0000 7f04d4e57640 1 -- 172.21.15.122:0/2722538775 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c80a5e30 con 0x7f04d006d350 2024-03-16T20:45:09.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.944+0000 7f04d80bd640 1 -- 172.21.15.122:0/2722538775 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04d006d350 msgr2=0x7f04d0151920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.944+0000 7f04d80bd640 1 --2- 172.21.15.122:0/2722538775 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04d006d350 0x7f04d0151920 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f04c8009d90 tx=0x7f04c801c970 comp rx=0 tx=0).stop 2024-03-16T20:45:09.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d80bd640 1 -- 172.21.15.122:0/2722538775 shutdown_connections 2024-03-16T20:45:09.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d80bd640 1 --2- 172.21.15.122:0/2722538775 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04d006d350 0x7f04d0151920 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d80bd640 1 --2- 172.21.15.122:0/2722538775 >> v2:172.21.15.122:3300/0 conn(0x7f04d0141580 0x7f04d00baac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d80bd640 1 -- 172.21.15.122:0/2722538775 >> 172.21.15.122:0/2722538775 conn(0x7f04d00bca20 msgr2=0x7f04d00bce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:09.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d80bd640 1 -- 172.21.15.122:0/2722538775 shutdown_connections 2024-03-16T20:45:09.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f029838b640 1 -- 172.21.15.122:0/3638449101 --> [v2:172.21.15.195:6824/3035467983,v1:172.21.15.195:6825/3035467983] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0278002cc0 con 0x7f027406f0c0 2024-03-16T20:45:09.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f0286ffd640 1 -- 172.21.15.122:0/3638449101 <== osd.7 v2:172.21.15.195:6824/3035467983 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0278002cc0 con 0x7f027406f0c0 2024-03-16T20:45:09.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d80bd640 1 -- 172.21.15.122:0/2722538775 wait complete. 2024-03-16T20:45:09.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d80bd640 1 Processor -- start 2024-03-16T20:45:09.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d80bd640 1 -- start start 2024-03-16T20:45:09.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d80bd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04d006d350 0x7f04d01ec6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d80bd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04d00bb000 0x7f04d0143560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d80bd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04d0141580 0x7f04d0143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d80bd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d00b4e00 con 0x7f04d0141580 2024-03-16T20:45:09.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d80bd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d01ecf60 con 0x7f04d00bb000 2024-03-16T20:45:09.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d80bd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d01ed240 con 0x7f04d006d350 2024-03-16T20:45:09.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d665a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04d0141580 0x7f04d0143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d665a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04d0141580 0x7f04d0143aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41358/0 (socket says 172.21.15.122:41358) 2024-03-16T20:45:09.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d665a640 1 -- 172.21.15.122:0/1490077840 learned_addr learned my addr 172.21.15.122:0/1490077840 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:09.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d5e59640 1 --2- 172.21.15.122:0/1490077840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04d006d350 0x7f04d01ec6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d5658640 1 --2- 172.21.15.122:0/1490077840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04d00bb000 0x7f04d0143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d5e59640 1 -- 172.21.15.122:0/1490077840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04d00bb000 msgr2=0x7f04d0143560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d5e59640 1 --2- 172.21.15.122:0/1490077840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04d00bb000 0x7f04d0143560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d5e59640 1 -- 172.21.15.122:0/1490077840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04d0141580 msgr2=0x7f04d0143aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d5e59640 1 --2- 172.21.15.122:0/1490077840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04d0141580 0x7f04d0143aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.948+0000 7f04d5e59640 1 -- 172.21.15.122:0/1490077840 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c8009a70 con 0x7f04d006d350 2024-03-16T20:45:09.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.952+0000 7f04d5e59640 1 --2- 172.21.15.122:0/1490077840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04d006d350 0x7f04d01ec6c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f04c000d5c0 tx=0x7f04c000da80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.952+0000 7f04c67fc640 1 -- 172.21.15.122:0/1490077840 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c00125f0 con 0x7f04d006d350 2024-03-16T20:45:09.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.952+0000 7f04d80bd640 1 -- 172.21.15.122:0/1490077840 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d00b67a0 con 0x7f04d006d350 2024-03-16T20:45:09.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.952+0000 7f04c67fc640 1 -- 172.21.15.122:0/1490077840 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c0012750 con 0x7f04d006d350 2024-03-16T20:45:09.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.952+0000 7f04c67fc640 1 -- 172.21.15.122:0/1490077840 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0012ac0 con 0x7f04d006d350 2024-03-16T20:45:09.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.952+0000 7f04d80bd640 1 -- 172.21.15.122:0/1490077840 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d01ecc00 con 0x7f04d006d350 2024-03-16T20:45:09.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.952+0000 7f04d80bd640 1 -- 172.21.15.122:0/1490077840 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f04d00b82b0 con 0x7f04d006d350 2024-03-16T20:45:09.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.952+0000 7f04c67fc640 1 -- 172.21.15.122:0/1490077840 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f04c0012c20 con 0x7f04d006d350 2024-03-16T20:45:09.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.952+0000 7f04c67fc640 1 --2- 172.21.15.122:0/1490077840 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f04b4066e40 0x7f04b4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.952+0000 7f04c67fc640 1 -- 172.21.15.122:0/1490077840 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04c0084fa0 con 0x7f04d006d350 2024-03-16T20:45:09.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.952+0000 7f04c67fc640 1 --2- 172.21.15.122:0/1490077840 >> [v2:172.21.15.195:6800/1568352129,v1:172.21.15.195:6801/1568352129] conn(0x7f04b406f0c0 0x7f04b40714e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:09.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.952+0000 7f04c67fc640 1 -- 172.21.15.122:0/1490077840 --> [v2:172.21.15.195:6800/1568352129,v1:172.21.15.195:6801/1568352129] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f04b4071b90 con 0x7f04b406f0c0 2024-03-16T20:45:09.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.952+0000 7f04c67fc640 1 -- 172.21.15.122:0/1490077840 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04c0085350 con 0x7f04d006d350 2024-03-16T20:45:09.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.952+0000 7f04d5658640 1 --2- 172.21.15.122:0/1490077840 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f04b4066e40 0x7f04b4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f04d665a640 1 --2- 172.21.15.122:0/1490077840 >> [v2:172.21.15.195:6800/1568352129,v1:172.21.15.195:6801/1568352129] conn(0x7f04b406f0c0 0x7f04b40714e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:09.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f23d5fe6640 1 -- 172.21.15.122:0/1664193591 --> [v2:172.21.15.122:6808/4193825355,v1:172.21.15.122:6809/4193825355] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f23d01623d0 con 0x7f23bc06efe0 2024-03-16T20:45:09.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f029838b640 1 -- 172.21.15.122:0/3638449101 >> [v2:172.21.15.195:6824/3035467983,v1:172.21.15.195:6825/3035467983] conn(0x7f027406f0c0 msgr2=0x7f02740714e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f029838b640 1 --2- 172.21.15.122:0/3638449101 >> [v2:172.21.15.195:6824/3035467983,v1:172.21.15.195:6825/3035467983] conn(0x7f027406f0c0 0x7f02740714e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f029838b640 1 -- 172.21.15.122:0/3638449101 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0274066e40 msgr2=0x7f0274069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f029838b640 1 --2- 172.21.15.122:0/3638449101 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0274066e40 0x7f0274069300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f028c001600 tx=0x7f028c009040 comp rx=0 tx=0).stop 2024-03-16T20:45:09.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f04d665a640 1 --2- 172.21.15.122:0/1490077840 >> [v2:172.21.15.195:6800/1568352129,v1:172.21.15.195:6801/1568352129] conn(0x7f04b406f0c0 0x7f04b40714e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f029838b640 1 -- 172.21.15.122:0/3638449101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f029014f900 msgr2=0x7f029008c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:09.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f029838b640 1 --2- 172.21.15.122:0/3638449101 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f029014f900 0x7f029008c100 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f028800cc30 tx=0x7f028800a5c0 comp rx=0 tx=0).stop 2024-03-16T20:45:09.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f04d5658640 1 --2- 172.21.15.122:0/1490077840 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f04b4066e40 0x7f04b4069300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f04d01f0a60 tx=0x7f04cc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:09.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f0296928640 1 -- 172.21.15.122:0/3638449101 reap_dead start 2024-03-16T20:45:09.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f029838b640 1 -- 172.21.15.122:0/3638449101 shutdown_connections 2024-03-16T20:45:09.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f029838b640 1 -- 172.21.15.122:0/3638449101 >> 172.21.15.122:0/3638449101 conn(0x7f02900b8d10 msgr2=0x7f0290099bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:09.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f23b7fff640 1 -- 172.21.15.122:0/1664193591 <== osd.0 v2:172.21.15.122:6808/4193825355 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f23d01623d0 con 0x7f23bc06efe0 2024-03-16T20:45:09.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f029838b640 1 -- 172.21.15.122:0/3638449101 shutdown_connections 2024-03-16T20:45:09.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f029838b640 1 -- 172.21.15.122:0/3638449101 wait complete. 2024-03-16T20:45:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f99ae1e5640 1 -- 172.21.15.122:0/3792156773 --> [v2:172.21.15.122:6816/362583029,v1:172.21.15.122:6817/362583029] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9988002d40 con 0x7f998c06f060 2024-03-16T20:45:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f9993fff640 1 -- 172.21.15.122:0/3792156773 <== osd.3 v2:172.21.15.122:6816/362583029 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9988002d40 con 0x7f998c06f060 2024-03-16T20:45:10.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f04c67fc640 1 -- 172.21.15.122:0/1490077840 <== osd.5 v2:172.21.15.195:6800/1568352129 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f04b4071b90 con 0x7f04b406f0c0 2024-03-16T20:45:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f99ae1e5640 1 -- 172.21.15.122:0/3792156773 >> [v2:172.21.15.122:6816/362583029,v1:172.21.15.122:6817/362583029] conn(0x7f998c06f060 msgr2=0x7f998c071480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f99ae1e5640 1 --2- 172.21.15.122:0/3792156773 >> [v2:172.21.15.122:6816/362583029,v1:172.21.15.122:6817/362583029] conn(0x7f998c06f060 0x7f998c071480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f99ae1e5640 1 -- 172.21.15.122:0/3792156773 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f998c066ec0 msgr2=0x7f998c069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f99ae1e5640 1 --2- 172.21.15.122:0/3792156773 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f998c066ec0 0x7f998c069380 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f99940b2000 tx=0x7f999400a040 comp rx=0 tx=0).stop 2024-03-16T20:45:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f99ae1e5640 1 -- 172.21.15.122:0/3792156773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99a00c8630 msgr2=0x7f99a01726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f99ae1e5640 1 --2- 172.21.15.122:0/3792156773 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f99a00c8630 0x7f99a01726d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f99a8068660 tx=0x7f99a8068a90 comp rx=0 tx=0).stop 2024-03-16T20:45:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f99a7fff640 1 -- 172.21.15.122:0/3792156773 reap_dead start 2024-03-16T20:45:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f99ae1e5640 1 -- 172.21.15.122:0/3792156773 shutdown_connections 2024-03-16T20:45:10.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f99ae1e5640 1 -- 172.21.15.122:0/3792156773 >> 172.21.15.122:0/3792156773 conn(0x7f99a003a890 msgr2=0x7f99a0006680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f99ae1e5640 1 -- 172.21.15.122:0/3792156773 shutdown_connections 2024-03-16T20:45:10.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f99ae1e5640 1 -- 172.21.15.122:0/3792156773 wait complete. 2024-03-16T20:45:10.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.956+0000 7f23d5fe6640 1 -- 172.21.15.122:0/1664193591 >> [v2:172.21.15.122:6808/4193825355,v1:172.21.15.122:6809/4193825355] conn(0x7f23bc06efe0 msgr2=0x7f23bc071400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.960+0000 7f23d5fe6640 1 --2- 172.21.15.122:0/1664193591 >> [v2:172.21.15.122:6808/4193825355,v1:172.21.15.122:6809/4193825355] conn(0x7f23bc06efe0 0x7f23bc071400 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7f23d5fe6640 1 -- 172.21.15.122:0/1664193591 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f23bc066e40 msgr2=0x7f23bc069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.002 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7f23d5fe6640 1 --2- 172.21.15.122:0/1664193591 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f23bc066e40 0x7f23bc069300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f23c0000c10 tx=0x7f23c0007910 comp rx=0 tx=0).stop 2024-03-16T20:45:10.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7fe004e30640 1 -- 172.21.15.122:0/3147738962 --> [v2:172.21.15.195:6810/1730924554,v1:172.21.15.195:6811/1730924554] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdfe4002d40 con 0x7fdfe806f5f0 2024-03-16T20:45:10.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7f23d5fe6640 1 -- 172.21.15.122:0/1664193591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23d00b4a40 msgr2=0x7f23d0097870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7f23d5fe6640 1 --2- 172.21.15.122:0/1664193591 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f23d00b4a40 0x7f23d0097870 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f23c40af000 tx=0x7f23c401d210 comp rx=0 tx=0).stop 2024-03-16T20:45:10.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7f23cffff640 1 -- 172.21.15.122:0/1664193591 reap_dead start 2024-03-16T20:45:10.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7f23d5fe6640 1 -- 172.21.15.122:0/1664193591 shutdown_connections 2024-03-16T20:45:10.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7f23d5fe6640 1 -- 172.21.15.122:0/1664193591 >> 172.21.15.122:0/1664193591 conn(0x7f23d009a4c0 msgr2=0x7f23d0152720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7f23d5fe6640 1 -- 172.21.15.122:0/1664193591 shutdown_connections 2024-03-16T20:45:10.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7fdfeeffd640 1 -- 172.21.15.122:0/3147738962 <== osd.6 v2:172.21.15.195:6810/1730924554 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdfe4002d40 con 0x7fdfe806f5f0 2024-03-16T20:45:10.004 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7f23d5fe6640 1 -- 172.21.15.122:0/1664193591 wait complete. 2024-03-16T20:45:10.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7fe004e30640 1 -- 172.21.15.122:0/3147738962 >> [v2:172.21.15.195:6810/1730924554,v1:172.21.15.195:6811/1730924554] conn(0x7fdfe806f5f0 msgr2=0x7fdfe8071a10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7fe004e30640 1 --2- 172.21.15.122:0/3147738962 >> [v2:172.21.15.195:6810/1730924554,v1:172.21.15.195:6811/1730924554] conn(0x7fdfe806f5f0 0x7fdfe8071a10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7fe004e30640 1 -- 172.21.15.122:0/3147738962 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fdfe8067370 msgr2=0x7fdfe8069830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7fe004e30640 1 --2- 172.21.15.122:0/3147738962 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fdfe8067370 0x7fdfe8069830 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe000153240 tx=0x7fdff0008040 comp rx=0 tx=0).stop 2024-03-16T20:45:10.005 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7fe004e30640 1 -- 172.21.15.122:0/3147738962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe000154520 msgr2=0x7fe0001e7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7fe004e30640 1 --2- 172.21.15.122:0/3147738962 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe000154520 0x7fe0001e7790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdff800c730 tx=0x7fdff800cbf0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7fdffed9d640 1 -- 172.21.15.122:0/3147738962 reap_dead start 2024-03-16T20:45:10.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7fe004e30640 1 -- 172.21.15.122:0/3147738962 shutdown_connections 2024-03-16T20:45:10.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7fe004e30640 1 -- 172.21.15.122:0/3147738962 >> 172.21.15.122:0/3147738962 conn(0x7fe00009de70 msgr2=0x7fe000150a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7fe004e30640 1 -- 172.21.15.122:0/3147738962 shutdown_connections 2024-03-16T20:45:10.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.964+0000 7fe004e30640 1 -- 172.21.15.122:0/3147738962 wait complete. 2024-03-16T20:45:10.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.968+0000 7fab84c74640 1 -- 172.21.15.122:0/2640283853 --> [v2:172.21.15.122:6800/685793600,v1:172.21.15.122:6801/685793600] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fab60002d40 con 0x7fab6c06f0b0 2024-03-16T20:45:10.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.968+0000 7fab67fff640 1 -- 172.21.15.122:0/2640283853 <== osd.1 v2:172.21.15.122:6800/685793600 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fab60002d40 con 0x7fab6c06f0b0 2024-03-16T20:45:10.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.968+0000 7f1e4be27640 1 -- 172.21.15.122:0/2554488820 --> [v2:172.21.15.195:6807/3059274194,v1:172.21.15.195:6809/3059274194] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1e441472e0 con 0x7f1e440b0ed0 2024-03-16T20:45:10.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.972+0000 7f1e3a7fc640 1 -- 172.21.15.122:0/2554488820 <== osd.4 v2:172.21.15.195:6807/3059274194 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1e34004040 con 0x7f1e440b0ed0 2024-03-16T20:45:10.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.972+0000 7f1e4be27640 1 -- 172.21.15.122:0/2554488820 >> [v2:172.21.15.195:6807/3059274194,v1:172.21.15.195:6809/3059274194] conn(0x7f1e440b0ed0 msgr2=0x7f1e4413dd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.972+0000 7f1e4be27640 1 --2- 172.21.15.122:0/2554488820 >> [v2:172.21.15.195:6807/3059274194,v1:172.21.15.195:6809/3059274194] conn(0x7f1e440b0ed0 0x7f1e4413dd30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.972+0000 7f1e4be27640 1 -- 172.21.15.122:0/2554488820 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1e28066e40 msgr2=0x7f1e28069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.972+0000 7f1e4be27640 1 --2- 172.21.15.122:0/2554488820 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1e28066e40 0x7f1e28069300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1e4406df90 tx=0x7f1e40009190 comp rx=0 tx=0).stop 2024-03-16T20:45:10.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.972+0000 7f1e4be27640 1 -- 172.21.15.122:0/2554488820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e440adad0 msgr2=0x7f1e44146390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.972+0000 7f1e4be27640 1 --2- 172.21.15.122:0/2554488820 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1e440adad0 0x7f1e44146390 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00cbf0 tx=0x7f1e3c00a430 comp rx=0 tx=0).stop 2024-03-16T20:45:10.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.972+0000 7f1e4a3c4640 1 -- 172.21.15.122:0/2554488820 reap_dead start 2024-03-16T20:45:10.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.972+0000 7f1e4be27640 1 -- 172.21.15.122:0/2554488820 shutdown_connections 2024-03-16T20:45:10.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.972+0000 7f1e4be27640 1 -- 172.21.15.122:0/2554488820 >> 172.21.15.122:0/2554488820 conn(0x7f1e440b96b0 msgr2=0x7f1e4406af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.972+0000 7f1e4be27640 1 -- 172.21.15.122:0/2554488820 shutdown_connections 2024-03-16T20:45:10.008 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.972+0000 7f1e4be27640 1 -- 172.21.15.122:0/2554488820 wait complete. 2024-03-16T20:45:10.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7ff81aa44640 1 -- 172.21.15.122:0/282025606 --> [v2:172.21.15.122:6819/2377812328,v1:172.21.15.122:6822/2377812328] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff80c16d5b0 con 0x7ff7f806f070 2024-03-16T20:45:10.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7ff810ff9640 1 -- 172.21.15.122:0/282025606 <== osd.2 v2:172.21.15.122:6819/2377812328 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff80c16d5b0 con 0x7ff7f806f070 2024-03-16T20:45:10.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7fab84c74640 1 -- 172.21.15.122:0/2640283853 >> [v2:172.21.15.122:6800/685793600,v1:172.21.15.122:6801/685793600] conn(0x7fab6c06f0b0 msgr2=0x7fab6c0714d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7fab84c74640 1 --2- 172.21.15.122:0/2640283853 >> [v2:172.21.15.122:6800/685793600,v1:172.21.15.122:6801/685793600] conn(0x7fab6c06f0b0 0x7fab6c0714d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7fab84c74640 1 -- 172.21.15.122:0/2640283853 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fab6c066e90 msgr2=0x7fab6c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7fab84c74640 1 --2- 172.21.15.122:0/2640283853 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fab6c066e90 0x7fab6c069350 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fab78005c10 tx=0x7fab78009040 comp rx=0 tx=0).stop 2024-03-16T20:45:10.009 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7fab84c74640 1 -- 172.21.15.122:0/2640283853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab800b8d00 msgr2=0x7fab8014a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7fab84c74640 1 --2- 172.21.15.122:0/2640283853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab800b8d00 0x7fab8014a500 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fab7400c410 tx=0x7fab7400c8d0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7fab7ffff640 1 -- 172.21.15.122:0/2640283853 reap_dead start 2024-03-16T20:45:10.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7fab84c74640 1 -- 172.21.15.122:0/2640283853 shutdown_connections 2024-03-16T20:45:10.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7fab84c74640 1 -- 172.21.15.122:0/2640283853 >> 172.21.15.122:0/2640283853 conn(0x7fab8006cd70 msgr2=0x7fab80074320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7fab84c74640 1 -- 172.21.15.122:0/2640283853 shutdown_connections 2024-03-16T20:45:10.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7fab84c74640 1 -- 172.21.15.122:0/2640283853 wait complete. 2024-03-16T20:45:10.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7f04d80bd640 1 -- 172.21.15.122:0/1490077840 --> [v2:172.21.15.195:6800/1568352129,v1:172.21.15.195:6801/1568352129] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f04d0151560 con 0x7f04b406f0c0 2024-03-16T20:45:10.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7ff81aa44640 1 -- 172.21.15.122:0/282025606 >> [v2:172.21.15.122:6819/2377812328,v1:172.21.15.122:6822/2377812328] conn(0x7ff7f806f070 msgr2=0x7ff7f8071490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7ff81aa44640 1 --2- 172.21.15.122:0/282025606 >> [v2:172.21.15.122:6819/2377812328,v1:172.21.15.122:6822/2377812328] conn(0x7ff7f806f070 0x7ff7f8071490 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7ff81aa44640 1 -- 172.21.15.122:0/282025606 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff7f8066e70 msgr2=0x7ff7f8069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7ff81aa44640 1 --2- 172.21.15.122:0/282025606 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff7f8066e70 0x7ff7f8069330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff80c005790 tx=0x7ff800008040 comp rx=0 tx=0).stop 2024-03-16T20:45:10.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7ff81aa44640 1 -- 172.21.15.122:0/282025606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff80c033d00 msgr2=0x7ff80c0c2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7ff81aa44640 1 --2- 172.21.15.122:0/282025606 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff80c033d00 0x7ff80c0c2f70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff814069ec0 tx=0x7ff814096450 comp rx=0 tx=0).stop 2024-03-16T20:45:10.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7ff818fe1640 1 -- 172.21.15.122:0/282025606 reap_dead start 2024-03-16T20:45:10.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7ff81aa44640 1 -- 172.21.15.122:0/282025606 shutdown_connections 2024-03-16T20:45:10.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7ff81aa44640 1 -- 172.21.15.122:0/282025606 >> 172.21.15.122:0/282025606 conn(0x7ff80c03a860 msgr2=0x7ff80c039010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7ff81aa44640 1 -- 172.21.15.122:0/282025606 shutdown_connections 2024-03-16T20:45:10.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7f04c67fc640 1 -- 172.21.15.122:0/1490077840 <== osd.5 v2:172.21.15.195:6800/1568352129 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f04d0151560 con 0x7f04b406f0c0 2024-03-16T20:45:10.012 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.976+0000 7ff81aa44640 1 -- 172.21.15.122:0/282025606 wait complete. 2024-03-16T20:45:10.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.980+0000 7f04b3fff640 1 -- 172.21.15.122:0/1490077840 >> [v2:172.21.15.195:6800/1568352129,v1:172.21.15.195:6801/1568352129] conn(0x7f04b406f0c0 msgr2=0x7f04b40714e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.980+0000 7f04b3fff640 1 --2- 172.21.15.122:0/1490077840 >> [v2:172.21.15.195:6800/1568352129,v1:172.21.15.195:6801/1568352129] conn(0x7f04b406f0c0 0x7f04b40714e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.980+0000 7f04b3fff640 1 -- 172.21.15.122:0/1490077840 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f04b4066e40 msgr2=0x7f04b4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.980+0000 7f04b3fff640 1 --2- 172.21.15.122:0/1490077840 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f04b4066e40 0x7f04b4069300 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f04d01f0a60 tx=0x7f04cc009040 comp rx=0 tx=0).stop 2024-03-16T20:45:10.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.980+0000 7f04b3fff640 1 -- 172.21.15.122:0/1490077840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04d006d350 msgr2=0x7f04d01ec6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.980+0000 7f04b3fff640 1 --2- 172.21.15.122:0/1490077840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f04d006d350 0x7f04d01ec6c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f04c000d5c0 tx=0x7f04c000da80 comp rx=0 tx=0).stop 2024-03-16T20:45:10.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.980+0000 7f04d665a640 1 -- 172.21.15.122:0/1490077840 reap_dead start 2024-03-16T20:45:10.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.980+0000 7f04b3fff640 1 -- 172.21.15.122:0/1490077840 shutdown_connections 2024-03-16T20:45:10.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.980+0000 7f04b3fff640 1 -- 172.21.15.122:0/1490077840 >> 172.21.15.122:0/1490077840 conn(0x7f04d00bca20 msgr2=0x7f04d00b8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.980+0000 7f04b3fff640 1 -- 172.21.15.122:0/1490077840 shutdown_connections 2024-03-16T20:45:10.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:09.980+0000 7f04b3fff640 1 -- 172.21.15.122:0/1490077840 wait complete. 2024-03-16T20:45:10.014 INFO:teuthology.orchestra.run.smithi122.stdout:55834574854 2024-03-16T20:45:10.015 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-16T20:45:10.015 INFO:teuthology.orchestra.run.smithi122.stdout:55834574854 2024-03-16T20:45:10.016 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-16T20:45:10.016 INFO:teuthology.orchestra.run.smithi122.stdout:55834574854 2024-03-16T20:45:10.017 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-16T20:45:10.017 INFO:teuthology.orchestra.run.smithi122.stdout:55834574854 2024-03-16T20:45:10.017 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-16T20:45:10.018 INFO:teuthology.orchestra.run.smithi122.stdout:55834574854 2024-03-16T20:45:10.018 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-16T20:45:10.019 INFO:teuthology.orchestra.run.smithi122.stdout:55834574855 2024-03-16T20:45:10.019 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-16T20:45:10.020 INFO:teuthology.orchestra.run.smithi122.stdout:55834574854 2024-03-16T20:45:10.021 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-16T20:45:10.021 INFO:teuthology.orchestra.run.smithi122.stdout:55834574855 2024-03-16T20:45:10.021 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-16T20:45:10.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f026b16e640 1 Processor -- start 2024-03-16T20:45:10.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f026b16e640 1 -- start start 2024-03-16T20:45:10.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f026b16e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f026413d590 0x7f02640b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f026b16e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026406d270 con 0x7f02640aed50 2024-03-16T20:45:10.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f026b16e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026406cc20 con 0x7f026406d580 2024-03-16T20:45:10.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f026b16e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f026406cf00 con 0x7f026413d590 2024-03-16T20:45:10.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f026a16c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f026413d590 0x7f02640b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f026a16c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f026413d590 0x7f02640b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41368/0 (socket says 172.21.15.122:41368) 2024-03-16T20:45:10.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f026a16c640 1 -- 172.21.15.122:0/3446642882 learned_addr learned my addr 172.21.15.122:0/3446642882 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:10.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f026996b640 1 --1- 172.21.15.122:0/3446642882 >> v1:172.21.15.195:6789/0 conn(0x7f026406d580 0x7f02640ae640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.122:51258/0 (socket says 172.21.15.122:51258) 2024-03-16T20:45:10.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f026a16c640 1 -- 172.21.15.122:0/3446642882 >> v1:172.21.15.195:6789/0 conn(0x7f026406d580 legacy=0x7f02640ae640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f026a16c640 1 -- 172.21.15.122:0/3446642882 >> v1:172.21.15.122:6789/0 conn(0x7f02640aed50 legacy=0x7f026414d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f026a16c640 1 -- 172.21.15.122:0/3446642882 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02640b9400 con 0x7f026413d590 2024-03-16T20:45:10.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f3aa2521640 1 Processor -- start 2024-03-16T20:45:10.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f51c4594640 1 Processor -- start 2024-03-16T20:45:10.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f51c4594640 1 -- start start 2024-03-16T20:45:10.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7fcd4540c640 1 Processor -- start 2024-03-16T20:45:10.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f5bd7c66640 1 Processor -- start 2024-03-16T20:45:10.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f5bd7c66640 1 -- start start 2024-03-16T20:45:10.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f95bd00a640 1 Processor -- start 2024-03-16T20:45:10.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f51c4594640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51bc0b6a10 0x7f51bc14be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f51c4594640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f51bc0b5610 0x7f51bc155c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f51c4594640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51bc14c3b0 con 0x7f51bc0b7600 2024-03-16T20:45:10.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f51c4594640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51bc14b9e0 con 0x7f51bc0b5610 2024-03-16T20:45:10.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f51c4594640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc14b3f0 con 0x7f51bc0b6a10 2024-03-16T20:45:10.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7fcd4540c640 1 -- start start 2024-03-16T20:45:10.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7fcd4540c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcd4014e7f0 0x7fcd40144e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7fcd4540c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd401453b0 0x7fcd40155b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7fcd4540c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd400b6770 con 0x7fcd400b6390 2024-03-16T20:45:10.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7fcd4540c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd400b5220 con 0x7fcd4014e7f0 2024-03-16T20:45:10.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7fcd4540c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd400b5490 con 0x7fcd401453b0 2024-03-16T20:45:10.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f51c2330640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51bc0b6a10 0x7f51bc14be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f51c2b31640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f51bc0b5610 0x7f51bc155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f51c2b31640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f51bc0b5610 0x7f51bc155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41376/0 (socket says 172.21.15.122:41376) 2024-03-16T20:45:10.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f51c2b31640 1 -- 172.21.15.122:0/1281594223 learned_addr learned my addr 172.21.15.122:0/1281594223 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:10.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7fcd3effd640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fcd400b6390 0x7fcd40071780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.122:51260/0 (socket says 172.21.15.122:51260) 2024-03-16T20:45:10.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7fcd3effd640 1 -- 172.21.15.122:0/1058456467 learned_addr learned my addr 172.21.15.122:0/1058456467 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:10.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f026a16c640 1 --2- 172.21.15.122:0/3446642882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f026413d590 0x7f02640b41c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f02600097f0 tx=0x7f026001c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6002cb012edb257 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.220+0000 7f3aa2521640 1 -- start start 2024-03-16T20:45:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f3aa2521640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3a9c0b8550 0x7f3a9c14eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f3aa2521640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f3a9c14f430 0x7f3a9c14f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f3aa2521640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9c0b41c0 con 0x7f3a9c0b3de0 2024-03-16T20:45:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f3aa2521640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9c0b9210 con 0x7f3a9c14f430 2024-03-16T20:45:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f3aa2521640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9c14fdc0 con 0x7f3a9c0b8550 2024-03-16T20:45:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f3a9bfff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f3a9c0b3de0 0x7f3a9c0b64a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58556/0 (socket says 172.21.15.122:58556) 2024-03-16T20:45:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f3a9bfff640 1 -- 172.21.15.122:0/3355951721 learned_addr learned my addr 172.21.15.122:0/3355951721 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f026916a640 1 -- 172.21.15.122:0/3446642882 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02600ac410 con 0x7f026413d590 2024-03-16T20:45:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f026916a640 1 -- 172.21.15.122:0/3446642882 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02600ac570 con 0x7f026413d590 2024-03-16T20:45:10.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f026916a640 1 -- 172.21.15.122:0/3446642882 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02600ac900 con 0x7f026413d590 2024-03-16T20:45:10.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f12b1e4e640 1 Processor -- start 2024-03-16T20:45:10.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f12b1e4e640 1 -- start start 2024-03-16T20:45:10.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f5bd7c66640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5bd00b4ee0 0x7f5bd013da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f5bd7c66640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd0072600 con 0x7f5bd00b6790 2024-03-16T20:45:10.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f95bd00a640 1 -- start start 2024-03-16T20:45:10.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f95bd00a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f95b8072220 0x7f95b8147260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f95bd00a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b80b4ee0 0x7f95b813da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f95bd00a640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b8072600 con 0x7f95b80b6790 2024-03-16T20:45:10.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f95bd00a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b80b5370 con 0x7f95b8072220 2024-03-16T20:45:10.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f95bd00a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b80b6270 con 0x7f95b80b4ee0 2024-03-16T20:45:10.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7fcd3dffb640 1 -- 172.21.15.122:0/1058456467 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652757584 0 0) 0x7fcd400b6770 con 0x7fcd400b6390 2024-03-16T20:45:10.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7fcd3dffb640 1 -- 172.21.15.122:0/1058456467 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd28003530 con 0x7fcd400b6390 2024-03-16T20:45:10.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f5bd5201640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5bd00b4ee0 0x7f5bd013da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f5bd7c66640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd00b5370 con 0x7f5bd0072220 2024-03-16T20:45:10.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f5bd5201640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5bd00b4ee0 0x7f5bd013da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.122:45158/0 (socket says 172.21.15.122:45158) 2024-03-16T20:45:10.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f5bd7c66640 1 -- 172.21.15.122:0/1561032008 --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd00b6270 con 0x7f5bd00b4ee0 2024-03-16T20:45:10.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f5bd5201640 1 -- 172.21.15.122:0/1561032008 learned_addr learned my addr 172.21.15.122:0/1561032008 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:10.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f95b659c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b80b4ee0 0x7f95b813da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f95b659c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b80b4ee0 0x7f95b813da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:34256/0 (socket says 172.21.15.122:34256) 2024-03-16T20:45:10.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f95b659c640 1 -- 172.21.15.122:0/3903175348 learned_addr learned my addr 172.21.15.122:0/3903175348 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:10.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f51c2330640 1 -- 172.21.15.122:0/1281594223 >> v2:172.21.15.122:3300/0 conn(0x7f51bc0b5610 msgr2=0x7f51bc155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:10.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f51c2330640 1 --2- 172.21.15.122:0/1281594223 >> v2:172.21.15.122:3300/0 conn(0x7f51bc0b5610 0x7f51bc155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f51c2330640 1 -- 172.21.15.122:0/1281594223 >> v1:172.21.15.195:6789/0 conn(0x7f51bc0b7600 legacy=0x7f51bc148470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f51c2330640 1 -- 172.21.15.122:0/1281594223 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51bc06e2a0 con 0x7f51bc0b6a10 2024-03-16T20:45:10.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f3aa0abe640 1 --2- 172.21.15.122:0/3355951721 >> v2:172.21.15.122:3300/0 conn(0x7f3a9c14f430 0x7f3a9c14f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7fcd3e7fc640 1 --2- 172.21.15.122:0/1058456467 >> v2:172.21.15.195:3300/0 conn(0x7fcd4014e7f0 0x7fcd40144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f026b16e640 1 -- 172.21.15.122:0/3446642882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f026413d590 msgr2=0x7f02640b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f026b16e640 1 --2- 172.21.15.122:0/3446642882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f026413d590 0x7f02640b41c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f02600097f0 tx=0x7f026001c390 comp rx=0 tx=0).stop 2024-03-16T20:45:10.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f026b16e640 1 -- 172.21.15.122:0/3446642882 shutdown_connections 2024-03-16T20:45:10.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f026b16e640 1 --2- 172.21.15.122:0/3446642882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f026413d590 0x7f02640b41c0 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f02600097f0 tx=0x7f026001c390 comp rx=0 tx=0).stop 2024-03-16T20:45:10.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f026b16e640 1 -- 172.21.15.122:0/3446642882 >> 172.21.15.122:0/3446642882 conn(0x7f026409a350 msgr2=0x7f026409a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f026b16e640 1 -- 172.21.15.122:0/3446642882 shutdown_connections 2024-03-16T20:45:10.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f026b16e640 1 -- 172.21.15.122:0/3446642882 wait complete. 2024-03-16T20:45:10.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f12b1e4e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12ac0b89f0 0x7f12ac142fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f12b1e4e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f12ac1434f0 0x7f12ac14d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f12b1e4e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ac0b8e30 con 0x7f12ac072f20 2024-03-16T20:45:10.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f12b1e4e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12ac143960 con 0x7f12ac1434f0 2024-03-16T20:45:10.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f12b1e4e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac073300 con 0x7f12ac0b89f0 2024-03-16T20:45:10.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f12abfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12ac0b89f0 0x7f12ac142fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f12abfff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12ac0b89f0 0x7f12ac142fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:34268/0 (socket says 172.21.15.122:34268) 2024-03-16T20:45:10.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f12abfff640 1 -- 172.21.15.122:0/651258564 learned_addr learned my addr 172.21.15.122:0/651258564 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:10.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f12b164d640 1 --2- 172.21.15.122:0/651258564 >> v2:172.21.15.195:3300/0 conn(0x7f12ac1434f0 0x7f12ac14d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f5bd5201640 1 -- 172.21.15.122:0/1561032008 >> v1:172.21.15.195:6789/0 conn(0x7f5bd0072220 legacy=0x7f5bd0147260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f5bd5201640 1 -- 172.21.15.122:0/1561032008 >> v1:172.21.15.122:6789/0 conn(0x7f5bd00b6790 legacy=0x7f5bd014e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f5bd5201640 1 -- 172.21.15.122:0/1561032008 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd00b5f30 con 0x7f5bd00b4ee0 2024-03-16T20:45:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f95b6d9d640 1 --2- 172.21.15.122:0/3903175348 >> v2:172.21.15.195:3300/0 conn(0x7f95b8072220 0x7f95b8147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f3a93fff640 1 --2- 172.21.15.122:0/3355951721 >> v2:172.21.15.195:3300/0 conn(0x7f3a9c0b8550 0x7f3a9c14eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f3a9b7fe640 1 -- 172.21.15.122:0/3355951721 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173091568 0 0) 0x7f3a9c0b41c0 con 0x7f3a9c0b3de0 2024-03-16T20:45:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f3a9b7fe640 1 -- 172.21.15.122:0/3355951721 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a88003530 con 0x7f3a9c0b3de0 2024-03-16T20:45:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f3aa0abe640 1 -- 172.21.15.122:0/3355951721 >> v2:172.21.15.195:3300/0 conn(0x7f3a9c0b8550 msgr2=0x7f3a9c14eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:10.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f3aa0abe640 1 --2- 172.21.15.122:0/3355951721 >> v2:172.21.15.195:3300/0 conn(0x7f3a9c0b8550 0x7f3a9c14eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f3aa0abe640 1 -- 172.21.15.122:0/3355951721 >> v1:172.21.15.122:6789/0 conn(0x7f3a9c0b3de0 legacy=0x7f3a9c0b64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f3aa0abe640 1 -- 172.21.15.122:0/3355951721 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c0b3380 con 0x7f3a9c14f430 2024-03-16T20:45:10.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7fcd3f7fe640 1 --2- 172.21.15.122:0/1058456467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd401453b0 0x7fcd40155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7fcd3e7fc640 1 -- 172.21.15.122:0/1058456467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd401453b0 msgr2=0x7fcd40155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:10.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7fcd3e7fc640 1 --2- 172.21.15.122:0/1058456467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd401453b0 0x7fcd40155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7fcd3e7fc640 1 -- 172.21.15.122:0/1058456467 >> v1:172.21.15.195:6789/0 conn(0x7fcd400b6390 legacy=0x7fcd40071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7fcd3e7fc640 1 -- 172.21.15.122:0/1058456467 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd400b40a0 con 0x7fcd4014e7f0 2024-03-16T20:45:10.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026b16e640 1 Processor -- start 2024-03-16T20:45:10.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026b16e640 1 -- start start 2024-03-16T20:45:10.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026b16e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f026406d580 0x7f0264143790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026b16e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02640aed50 0x7f026413ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026b16e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f026413ef40 0x7f0264071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026b16e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f026413df50 con 0x7f02640aed50 2024-03-16T20:45:10.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026b16e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02641408c0 con 0x7f026406d580 2024-03-16T20:45:10.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026b16e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0264140af0 con 0x7f026413ef40 2024-03-16T20:45:10.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026996b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02640aed50 0x7f026413ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026996b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02640aed50 0x7f026413ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41402/0 (socket says 172.21.15.122:41402) 2024-03-16T20:45:10.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026996b640 1 -- 172.21.15.122:0/2266863632 learned_addr learned my addr 172.21.15.122:0/2266863632 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:10.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026a96d640 1 --2- 172.21.15.122:0/2266863632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f026413ef40 0x7f0264071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f95b5d9b640 1 -- 172.21.15.122:0/3903175348 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114521987 0 0) 0x7f95b8072600 con 0x7f95b80b6790 2024-03-16T20:45:10.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f95b5d9b640 1 -- 172.21.15.122:0/3903175348 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95a0003530 con 0x7f95b80b6790 2024-03-16T20:45:10.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f95b659c640 1 -- 172.21.15.122:0/3903175348 >> v2:172.21.15.195:3300/0 conn(0x7f95b8072220 msgr2=0x7f95b8147260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:10.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f95b659c640 1 --2- 172.21.15.122:0/3903175348 >> v2:172.21.15.195:3300/0 conn(0x7f95b8072220 0x7f95b8147260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f95b659c640 1 -- 172.21.15.122:0/3903175348 >> v1:172.21.15.122:6789/0 conn(0x7f95b80b6790 legacy=0x7f95b8070cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f95b659c640 1 -- 172.21.15.122:0/3903175348 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b81477a0 con 0x7f95b80b4ee0 2024-03-16T20:45:10.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f95b6d9d640 1 --2- 172.21.15.122:0/3903175348 >> v2:172.21.15.195:3300/0 conn(0x7f95b8072220 0x7f95b8147260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:10.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f5bd5201640 1 --2- 172.21.15.122:0/1561032008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bd00b4ee0 0x7f5bd013da50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009870 tx=0x7f5bcc01e6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=579b37e6407a7c97 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f51c2330640 1 --2- 172.21.15.122:0/1281594223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51bc0b6a10 0x7f51bc14be70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f51b4009d90 tx=0x7f51b401c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e8b839458a9b978 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f51c132e640 1 -- 172.21.15.122:0/1281594223 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b40a5910 con 0x7f51bc0b6a10 2024-03-16T20:45:10.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f51c132e640 1 -- 172.21.15.122:0/1281594223 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b40a5a70 con 0x7f51bc0b6a10 2024-03-16T20:45:10.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f51c132e640 1 -- 172.21.15.122:0/1281594223 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b40a5e00 con 0x7f51bc0b6a10 2024-03-16T20:45:10.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f12abfff640 1 -- 172.21.15.122:0/651258564 >> v2:172.21.15.195:3300/0 conn(0x7f12ac1434f0 msgr2=0x7f12ac14d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:10.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f12abfff640 1 --2- 172.21.15.122:0/651258564 >> v2:172.21.15.195:3300/0 conn(0x7f12ac1434f0 0x7f12ac14d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f12abfff640 1 -- 172.21.15.122:0/651258564 >> v1:172.21.15.122:6789/0 conn(0x7f12ac072f20 legacy=0x7f12ac09d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.224+0000 7f12abfff640 1 -- 172.21.15.122:0/651258564 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ac0aef50 con 0x7f12ac0b89f0 2024-03-16T20:45:10.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f2aa60cb640 1 Processor -- start 2024-03-16T20:45:10.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f2aa60cb640 1 -- start start 2024-03-16T20:45:10.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f2aa60cb640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2aa00b80c0 0x7f2aa0157410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f2aa60cb640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f2aa00b4a40 0x7f2aa0153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f2aa60cb640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa00b4ef0 con 0x7f2aa00b7ce0 2024-03-16T20:45:10.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f2aa60cb640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa01580f0 con 0x7f2aa00b4a40 2024-03-16T20:45:10.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f2aa60cb640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa0157a30 con 0x7f2aa00b80c0 2024-03-16T20:45:10.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f2a9effd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2aa00b80c0 0x7f2aa0157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f3aa0abe640 1 --2- 172.21.15.122:0/3355951721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a9c14f430 0x7f3a9c14f880 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c072d70 tx=0x7f3a9401c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed9e9df8dacb30e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026996b640 1 -- 172.21.15.122:0/2266863632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f026413ef40 msgr2=0x7f0264071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026996b640 1 --2- 172.21.15.122:0/2266863632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f026413ef40 0x7f0264071590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026996b640 1 -- 172.21.15.122:0/2266863632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f026406d580 msgr2=0x7f0264143790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026996b640 1 --2- 172.21.15.122:0/2266863632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f026406d580 0x7f0264143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026996b640 1 -- 172.21.15.122:0/2266863632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02600094d0 con 0x7f02640aed50 2024-03-16T20:45:10.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f2a9effd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2aa00b80c0 0x7f2aa0157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.122:45192/0 (socket says 172.21.15.122:45192) 2024-03-16T20:45:10.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f2a9effd640 1 -- 172.21.15.122:0/2780222227 learned_addr learned my addr 172.21.15.122:0/2780222227 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:10.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f2a9ffff640 1 --2- 172.21.15.122:0/2780222227 >> v2:172.21.15.122:3300/0 conn(0x7f2aa00b4a40 0x7f2aa0153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f5bd4a00640 1 -- 172.21.15.122:0/1561032008 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0b2600 con 0x7f5bd00b4ee0 2024-03-16T20:45:10.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f5bd4a00640 1 -- 172.21.15.122:0/1561032008 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc0b2760 con 0x7f5bd00b4ee0 2024-03-16T20:45:10.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f5bd4a00640 1 -- 172.21.15.122:0/1561032008 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0b2af0 con 0x7f5bd00b4ee0 2024-03-16T20:45:10.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7fcd3e7fc640 1 --2- 172.21.15.122:0/1058456467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd4014e7f0 0x7fcd40144e70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcd340097c0 tx=0x7fcd3401c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15e0db7a634f7901 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f51c4594640 1 -- 172.21.15.122:0/1281594223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51bc0b6a10 msgr2=0x7f51bc14be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f51c4594640 1 --2- 172.21.15.122:0/1281594223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51bc0b6a10 0x7f51bc14be70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f51b4009d90 tx=0x7f51b401c850 comp rx=0 tx=0).stop 2024-03-16T20:45:10.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7fcd3dffb640 1 -- 172.21.15.122:0/1058456467 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34004230 con 0x7fcd4014e7f0 2024-03-16T20:45:10.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f51c4594640 1 -- 172.21.15.122:0/1281594223 shutdown_connections 2024-03-16T20:45:10.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f51c4594640 1 --2- 172.21.15.122:0/1281594223 >> v2:172.21.15.122:3300/0 conn(0x7f51bc0b5610 0x7f51bc155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f51c4594640 1 --2- 172.21.15.122:0/1281594223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51bc0b6a10 0x7f51bc14be70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f51c4594640 1 -- 172.21.15.122:0/1281594223 >> 172.21.15.122:0/1281594223 conn(0x7f51bc09de70 msgr2=0x7f51bc0b86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026996b640 1 --2- 172.21.15.122:0/2266863632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02640aed50 0x7f026413ea00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f025c00e450 tx=0x7f025c00e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7fcd3dffb640 1 -- 172.21.15.122:0/1058456467 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd340a7050 con 0x7fcd4014e7f0 2024-03-16T20:45:10.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7fcd3dffb640 1 -- 172.21.15.122:0/1058456467 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34003620 con 0x7fcd4014e7f0 2024-03-16T20:45:10.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f2a9effd640 1 -- 172.21.15.122:0/2780222227 >> v2:172.21.15.122:3300/0 conn(0x7f2aa00b4a40 msgr2=0x7f2aa0153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:10.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f2a9effd640 1 --2- 172.21.15.122:0/2780222227 >> v2:172.21.15.122:3300/0 conn(0x7f2aa00b4a40 0x7f2aa0153ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f2a9effd640 1 -- 172.21.15.122:0/2780222227 >> v1:172.21.15.122:6789/0 conn(0x7f2aa00b7ce0 legacy=0x7f2aa0095180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:10.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f025affd640 1 -- 172.21.15.122:0/2266863632 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c006070 con 0x7f02640aed50 2024-03-16T20:45:10.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f025affd640 1 -- 172.21.15.122:0/2266863632 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025c0061d0 con 0x7f02640aed50 2024-03-16T20:45:10.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f51c4594640 1 -- 172.21.15.122:0/1281594223 shutdown_connections 2024-03-16T20:45:10.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f95b659c640 1 --2- 172.21.15.122:0/3903175348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b80b4ee0 0x7f95b813da50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f95b000d150 tx=0x7f95b001ebc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52792eaad9ea59a0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f12abfff640 1 --2- 172.21.15.122:0/651258564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12ac0b89f0 0x7f12ac142fb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f12a000a910 tx=0x7f12a001c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44c55b30deb77aa8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f2a9effd640 1 -- 172.21.15.122:0/2780222227 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa0157d10 con 0x7f2aa00b80c0 2024-03-16T20:45:10.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f2a9ffff640 1 --2- 172.21.15.122:0/2780222227 >> v2:172.21.15.122:3300/0 conn(0x7f2aa00b4a40 0x7f2aa0153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:45:10.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f95b5d9b640 1 -- 172.21.15.122:0/3903175348 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b000b2d0 con 0x7f95b80b4ee0 2024-03-16T20:45:10.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f95b5d9b640 1 -- 172.21.15.122:0/3903175348 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95b000b430 con 0x7f95b80b4ee0 2024-03-16T20:45:10.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f95b5d9b640 1 -- 172.21.15.122:0/3903175348 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95b00a7cb0 con 0x7f95b80b4ee0 2024-03-16T20:45:10.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f3a9b7fe640 1 -- 172.21.15.122:0/3355951721 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a940a5840 con 0x7f3a9c14f430 2024-03-16T20:45:10.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f3a9b7fe640 1 -- 172.21.15.122:0/3355951721 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a940a59a0 con 0x7f3a9c14f430 2024-03-16T20:45:10.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f3a9b7fe640 1 -- 172.21.15.122:0/3355951721 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a940a5840 con 0x7f3a9c14f430 2024-03-16T20:45:10.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f5bd7c66640 1 -- 172.21.15.122:0/1561032008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bd00b4ee0 msgr2=0x7f5bd013da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f5bd7c66640 1 --2- 172.21.15.122:0/1561032008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bd00b4ee0 0x7f5bd013da50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009870 tx=0x7f5bcc01e6a0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7fcd4540c640 1 -- 172.21.15.122:0/1058456467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd4014e7f0 msgr2=0x7fcd40144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7fcd4540c640 1 --2- 172.21.15.122:0/1058456467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd4014e7f0 0x7fcd40144e70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcd340097c0 tx=0x7fcd3401c350 comp rx=0 tx=0).stop 2024-03-16T20:45:10.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f5bd7c66640 1 -- 172.21.15.122:0/1561032008 shutdown_connections 2024-03-16T20:45:10.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f5bd7c66640 1 --2- 172.21.15.122:0/1561032008 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bd00b4ee0 0x7f5bd013da50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f5bd7c66640 1 -- 172.21.15.122:0/1561032008 >> 172.21.15.122:0/1561032008 conn(0x7f5bd006c0a0 msgr2=0x7f5bd006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7fcd4540c640 1 -- 172.21.15.122:0/1058456467 shutdown_connections 2024-03-16T20:45:10.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7fcd4540c640 1 --2- 172.21.15.122:0/1058456467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd401453b0 0x7fcd40155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7fcd4540c640 1 --2- 172.21.15.122:0/1058456467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd4014e7f0 0x7fcd40144e70 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcd340097c0 tx=0x7fcd3401c350 comp rx=0 tx=0).stop 2024-03-16T20:45:10.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7fcd4540c640 1 -- 172.21.15.122:0/1058456467 >> 172.21.15.122:0/1058456467 conn(0x7fcd4006c0a0 msgr2=0x7fcd4006c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026b16e640 1 -- 172.21.15.122:0/2266863632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f026413f3d0 con 0x7f02640aed50 2024-03-16T20:45:10.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f025affd640 1 -- 172.21.15.122:0/2266863632 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025c006540 con 0x7f02640aed50 2024-03-16T20:45:10.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f026b16e640 1 -- 172.21.15.122:0/2266863632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026413e4e0 con 0x7f02640aed50 2024-03-16T20:45:10.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd4540c640 1 -- 172.21.15.122:0/1058456467 shutdown_connections 2024-03-16T20:45:10.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.228+0000 7f51c4594640 1 -- 172.21.15.122:0/1281594223 wait complete. 2024-03-16T20:45:10.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f026b16e640 1 -- 172.21.15.122:0/2266863632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026413e700 con 0x7f02640aed50 2024-03-16T20:45:10.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12ab7fe640 1 -- 172.21.15.122:0/651258564 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a00a5950 con 0x7f12ac0b89f0 2024-03-16T20:45:10.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12ab7fe640 1 -- 172.21.15.122:0/651258564 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a00a9070 con 0x7f12ac0b89f0 2024-03-16T20:45:10.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12ab7fe640 1 -- 172.21.15.122:0/651258564 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a00a5cd0 con 0x7f12ac0b89f0 2024-03-16T20:45:10.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c4594640 1 Processor -- start 2024-03-16T20:45:10.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c4594640 1 -- start start 2024-03-16T20:45:10.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95bd00a640 1 -- 172.21.15.122:0/3903175348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b80b4ee0 msgr2=0x7f95b813da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95bd00a640 1 --2- 172.21.15.122:0/3903175348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b80b4ee0 0x7f95b813da50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f95b000d150 tx=0x7f95b001ebc0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95bd00a640 1 -- 172.21.15.122:0/3903175348 shutdown_connections 2024-03-16T20:45:10.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95bd00a640 1 --2- 172.21.15.122:0/3903175348 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b80b4ee0 0x7f95b813da50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95bd00a640 1 --2- 172.21.15.122:0/3903175348 >> v2:172.21.15.195:3300/0 conn(0x7f95b8072220 0x7f95b8147260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95bd00a640 1 -- 172.21.15.122:0/3903175348 >> 172.21.15.122:0/3903175348 conn(0x7f95b806c0a0 msgr2=0x7f95b806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95bd00a640 1 -- 172.21.15.122:0/3903175348 shutdown_connections 2024-03-16T20:45:10.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f3aa2521640 1 -- 172.21.15.122:0/3355951721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a9c14f430 msgr2=0x7f3a9c14f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f3aa2521640 1 --2- 172.21.15.122:0/3355951721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a9c14f430 0x7f3a9c14f880 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c072d70 tx=0x7f3a9401c870 comp rx=0 tx=0).stop 2024-03-16T20:45:10.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f3aa2521640 1 -- 172.21.15.122:0/3355951721 shutdown_connections 2024-03-16T20:45:10.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f3aa2521640 1 --2- 172.21.15.122:0/3355951721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a9c14f430 0x7f3a9c14f880 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f3aa2521640 1 --2- 172.21.15.122:0/3355951721 >> v2:172.21.15.195:3300/0 conn(0x7f3a9c0b8550 0x7f3a9c14eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f3aa2521640 1 -- 172.21.15.122:0/3355951721 >> 172.21.15.122:0/3355951721 conn(0x7f3a9c06de30 msgr2=0x7f3a9c06c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f3aa2521640 1 -- 172.21.15.122:0/3355951721 shutdown_connections 2024-03-16T20:45:10.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f025affd640 1 -- 172.21.15.122:0/2266863632 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f025c0066a0 con 0x7f02640aed50 2024-03-16T20:45:10.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f025affd640 1 --2- 172.21.15.122:0/2266863632 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0248066e40 0x7f0248069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f025affd640 1 -- 172.21.15.122:0/2266863632 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f025c087600 con 0x7f02640aed50 2024-03-16T20:45:10.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12b1e4e640 1 -- 172.21.15.122:0/651258564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12ac0b89f0 msgr2=0x7f12ac142fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12b1e4e640 1 --2- 172.21.15.122:0/651258564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12ac0b89f0 0x7f12ac142fb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f12a000a910 tx=0x7f12a001c8b0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12b1e4e640 1 -- 172.21.15.122:0/651258564 shutdown_connections 2024-03-16T20:45:10.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12b1e4e640 1 --2- 172.21.15.122:0/651258564 >> v2:172.21.15.195:3300/0 conn(0x7f12ac1434f0 0x7f12ac14d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12b1e4e640 1 --2- 172.21.15.122:0/651258564 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12ac0b89f0 0x7f12ac142fb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12b1e4e640 1 -- 172.21.15.122:0/651258564 >> 172.21.15.122:0/651258564 conn(0x7f12ac099c20 msgr2=0x7f12ac06c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12b1e4e640 1 -- 172.21.15.122:0/651258564 shutdown_connections 2024-03-16T20:45:10.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12b1e4e640 1 -- 172.21.15.122:0/651258564 wait complete. 2024-03-16T20:45:10.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12b1e4e640 1 Processor -- start 2024-03-16T20:45:10.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12b1e4e640 1 -- start start 2024-03-16T20:45:10.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c4594640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51bc0b5610 0x7f51bc0b0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c4594640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51bc0b6a10 0x7f51bc0b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c4594640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51bc0b7600 0x7f51bc1e46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c4594640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc0b13b0 con 0x7f51bc0b5610 2024-03-16T20:45:10.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c4594640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc0b1690 con 0x7f51bc0b7600 2024-03-16T20:45:10.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c4594640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51bc0b1970 con 0x7f51bc0b6a10 2024-03-16T20:45:10.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c1b2f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51bc0b6a10 0x7f51bc0b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c1b2f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51bc0b6a10 0x7f51bc0b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:34272/0 (socket says 172.21.15.122:34272) 2024-03-16T20:45:10.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c1b2f640 1 -- 172.21.15.122:0/1881626051 learned_addr learned my addr 172.21.15.122:0/1881626051 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:10.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c2b31640 1 --2- 172.21.15.122:0/1881626051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51bc0b7600 0x7f51bc1e46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c2330640 1 --2- 172.21.15.122:0/1881626051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51bc0b5610 0x7f51bc0b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c1b2f640 1 -- 172.21.15.122:0/1881626051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51bc0b7600 msgr2=0x7f51bc1e46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c1b2f640 1 --2- 172.21.15.122:0/1881626051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51bc0b7600 0x7f51bc1e46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c1b2f640 1 -- 172.21.15.122:0/1881626051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51bc0b5610 msgr2=0x7f51bc0b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c1b2f640 1 --2- 172.21.15.122:0/1881626051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51bc0b5610 0x7f51bc0b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c1b2f640 1 -- 172.21.15.122:0/1881626051 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b4009a70 con 0x7f51bc0b6a10 2024-03-16T20:45:10.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f51c1b2f640 1 --2- 172.21.15.122:0/1881626051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51bc0b6a10 0x7f51bc0b1c70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00c390 tx=0x7f51ac00c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95bd00a640 1 -- 172.21.15.122:0/3903175348 wait complete. 2024-03-16T20:45:10.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95bd00a640 1 Processor -- start 2024-03-16T20:45:10.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95bd00a640 1 -- start start 2024-03-16T20:45:10.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95bd00a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8072220 0x7f95b81e2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95bd00a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b80b6790 0x7f95b81f0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95bd00a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95b81f0570 0x7f95b81dd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95b6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8072220 0x7f95b81e2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95b6d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8072220 0x7f95b81e2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41442/0 (socket says 172.21.15.122:41442) 2024-03-16T20:45:10.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95b6d9d640 1 -- 172.21.15.122:0/209578088 learned_addr learned my addr 172.21.15.122:0/209578088 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:10.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95bd00a640 1 -- 172.21.15.122:0/209578088 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b81f0a20 con 0x7f95b8072220 2024-03-16T20:45:10.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f3aa2521640 1 -- 172.21.15.122:0/3355951721 wait complete. 2024-03-16T20:45:10.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f3aa2521640 1 Processor -- start 2024-03-16T20:45:10.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd4540c640 1 -- 172.21.15.122:0/1058456467 wait complete. 2024-03-16T20:45:10.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd4540c640 1 Processor -- start 2024-03-16T20:45:10.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd4540c640 1 -- start start 2024-03-16T20:45:10.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd4540c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd400b6390 0x7fcd400b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd4540c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd401453b0 0x7fcd400b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd4540c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd40154990 0x7fcd40154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd4540c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd40150430 con 0x7fcd40154990 2024-03-16T20:45:10.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd4540c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd400b0c80 con 0x7fcd400b6390 2024-03-16T20:45:10.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd4540c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd40139880 con 0x7fcd401453b0 2024-03-16T20:45:10.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd3f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd40154990 0x7fcd40154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd3f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd40154990 0x7fcd40154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41432/0 (socket says 172.21.15.122:41432) 2024-03-16T20:45:10.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd3f7fe640 1 -- 172.21.15.122:0/2621859215 learned_addr learned my addr 172.21.15.122:0/2621859215 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:10.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd3f7fe640 1 -- 172.21.15.122:0/2621859215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd401453b0 msgr2=0x7fcd400b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd3e7fc640 1 --2- 172.21.15.122:0/2621859215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd401453b0 0x7fcd400b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd3f7fe640 1 --2- 172.21.15.122:0/2621859215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd401453b0 0x7fcd400b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd3f7fe640 1 -- 172.21.15.122:0/2621859215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd400b6390 msgr2=0x7fcd400b2550 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-16T20:45:10.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd3f7fe640 1 --2- 172.21.15.122:0/2621859215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd400b6390 0x7fcd400b2550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd3f7fe640 1 -- 172.21.15.122:0/2621859215 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd34009420 con 0x7fcd40154990 2024-03-16T20:45:10.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7fcd3f7fe640 1 --2- 172.21.15.122:0/2621859215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd40154990 0x7fcd40154e10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000d800 tx=0x7fcd3000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.263 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95bd00a640 1 -- 172.21.15.122:0/209578088 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b80b2da0 con 0x7f95b81f0570 2024-03-16T20:45:10.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95bd00a640 1 -- 172.21.15.122:0/209578088 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b80b3080 con 0x7f95b80b6790 2024-03-16T20:45:10.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f026a16c640 1 --2- 172.21.15.122:0/2266863632 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0248066e40 0x7f0248069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12b1e4e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12ac072f20 0x7f12ac1e2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.264 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95b759e640 1 --2- 172.21.15.122:0/209578088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95b81f0570 0x7f95b81dd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12b1e4e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12ac0b89f0 0x7f12ac1efd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12b1e4e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12ac1434f0 0x7f12ac1ec3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f2a9effd640 1 --2- 172.21.15.122:0/2780222227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aa00b80c0 0x7f2aa0157410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2a94009a60 tx=0x7f2a9401c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1877c0ea5a561e09 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f2a9e7fc640 1 -- 172.21.15.122:0/2780222227 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94004030 con 0x7f2aa00b80c0 2024-03-16T20:45:10.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f2a9e7fc640 1 -- 172.21.15.122:0/2780222227 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a94004190 con 0x7f2aa00b80c0 2024-03-16T20:45:10.265 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f95b659c640 1 --2- 172.21.15.122:0/209578088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b80b6790 0x7f95b81f0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12b1e4e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac070a90 con 0x7f12ac0b89f0 2024-03-16T20:45:10.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12b1e4e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac070d70 con 0x7f12ac1434f0 2024-03-16T20:45:10.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f12b1e4e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac06fe60 con 0x7f12ac072f20 2024-03-16T20:45:10.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f2a9e7fc640 1 -- 172.21.15.122:0/2780222227 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94003620 con 0x7f2aa00b80c0 2024-03-16T20:45:10.266 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f12abfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12ac0b89f0 0x7f12ac1efd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f3aa2521640 1 -- start start 2024-03-16T20:45:10.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f3aa2521640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a9c0b3de0 0x7f3a9c0af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f3aa2521640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a9c0b8550 0x7f3a9c08d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.232+0000 7f3aa2521640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a9c14f430 0x7f3a9c08d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3aa2521640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c14bf70 con 0x7f3a9c14f430 2024-03-16T20:45:10.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3aa2521640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0b10e0 con 0x7f3a9c0b3de0 2024-03-16T20:45:10.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3aa2521640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c14edb0 con 0x7f3a9c0b8550 2024-03-16T20:45:10.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f12abfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12ac0b89f0 0x7f12ac1efd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41456/0 (socket says 172.21.15.122:41456) 2024-03-16T20:45:10.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f12abfff640 1 -- 172.21.15.122:0/1132671889 learned_addr learned my addr 172.21.15.122:0/1132671889 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:10.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f51b2ffd640 1 -- 172.21.15.122:0/1881626051 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac0052d0 con 0x7f51bc0b6a10 2024-03-16T20:45:10.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f51b2ffd640 1 -- 172.21.15.122:0/1881626051 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac005430 con 0x7f51bc0b6a10 2024-03-16T20:45:10.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f51b2ffd640 1 -- 172.21.15.122:0/1881626051 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac003820 con 0x7f51bc0b6a10 2024-03-16T20:45:10.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f51c4594640 1 -- 172.21.15.122:0/1881626051 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51bc0b21b0 con 0x7f51bc0b6a10 2024-03-16T20:45:10.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f51c4594640 1 -- 172.21.15.122:0/1881626051 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51bc0733d0 con 0x7f51bc0b6a10 2024-03-16T20:45:10.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f95b6d9d640 1 -- 172.21.15.122:0/209578088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b80b6790 msgr2=0x7f95b81f0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f95b6d9d640 1 --2- 172.21.15.122:0/209578088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b80b6790 0x7f95b81f0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f95b6d9d640 1 -- 172.21.15.122:0/209578088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95b81f0570 msgr2=0x7f95b81dd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f95b6d9d640 1 --2- 172.21.15.122:0/209578088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95b81f0570 0x7f95b81dd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f95b6d9d640 1 -- 172.21.15.122:0/209578088 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b000bd40 con 0x7f95b8072220 2024-03-16T20:45:10.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f95b659c640 1 --2- 172.21.15.122:0/209578088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b80b6790 0x7f95b81f0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:10.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3a93fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a9c0b8550 0x7f3a9c08d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3a93fff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a9c0b8550 0x7f3a9c08d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:34304/0 (socket says 172.21.15.122:34304) 2024-03-16T20:45:10.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3a93fff640 1 -- 172.21.15.122:0/322907007 learned_addr learned my addr 172.21.15.122:0/322907007 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:10.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3a9bfff640 1 --2- 172.21.15.122:0/322907007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a9c0b3de0 0x7f3a9c0af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2aa60cb640 1 -- 172.21.15.122:0/2780222227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aa00b80c0 msgr2=0x7f2aa0157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2aa60cb640 1 --2- 172.21.15.122:0/2780222227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aa00b80c0 0x7f2aa0157410 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2a94009a60 tx=0x7f2a9401c3d0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f95b6d9d640 1 --2- 172.21.15.122:0/209578088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8072220 0x7f95b81e2420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00c7c0 tx=0x7f95ac00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f12abfff640 1 -- 172.21.15.122:0/1132671889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12ac072f20 msgr2=0x7f12ac1e2330 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-16T20:45:10.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f12abfff640 1 --2- 172.21.15.122:0/1132671889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12ac072f20 0x7f12ac1e2330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f12abfff640 1 -- 172.21.15.122:0/1132671889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12ac1434f0 msgr2=0x7f12ac1ec3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-16T20:45:10.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f12abfff640 1 --2- 172.21.15.122:0/1132671889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12ac1434f0 0x7f12ac1ec3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f12abfff640 1 -- 172.21.15.122:0/1132671889 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a000a570 con 0x7f12ac0b89f0 2024-03-16T20:45:10.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2aa60cb640 1 -- 172.21.15.122:0/2780222227 shutdown_connections 2024-03-16T20:45:10.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2aa60cb640 1 --2- 172.21.15.122:0/2780222227 >> v2:172.21.15.122:3300/0 conn(0x7f2aa00b4a40 0x7f2aa0153ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2aa60cb640 1 --2- 172.21.15.122:0/2780222227 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aa00b80c0 0x7f2aa0157410 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2aa60cb640 1 -- 172.21.15.122:0/2780222227 >> 172.21.15.122:0/2780222227 conn(0x7f2aa009a4c0 msgr2=0x7f2aa009a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2aa60cb640 1 -- 172.21.15.122:0/2780222227 shutdown_connections 2024-03-16T20:45:10.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f12abfff640 1 --2- 172.21.15.122:0/1132671889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12ac0b89f0 0x7f12ac1efd30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f12a001af60 tx=0x7f12a0019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2aa60cb640 1 -- 172.21.15.122:0/2780222227 wait complete. 2024-03-16T20:45:10.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3a9bfff640 1 -- 172.21.15.122:0/322907007 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a9c0b8550 msgr2=0x7f3a9c08d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3a9bfff640 1 --2- 172.21.15.122:0/322907007 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a9c0b8550 0x7f3a9c08d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3a9bfff640 1 -- 172.21.15.122:0/322907007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a9c14f430 msgr2=0x7f3a9c08d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3a9bfff640 1 --2- 172.21.15.122:0/322907007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a9c14f430 0x7f3a9c08d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3a9bfff640 1 -- 172.21.15.122:0/322907007 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a94009a70 con 0x7f3a9c0b3de0 2024-03-16T20:45:10.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f5bd7c66640 1 -- 172.21.15.122:0/1561032008 shutdown_connections 2024-03-16T20:45:10.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f5bd7c66640 1 -- 172.21.15.122:0/1561032008 wait complete. 2024-03-16T20:45:10.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3a9bfff640 1 --2- 172.21.15.122:0/322907007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a9c0b3de0 0x7f3a9c0af550 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400c430 tx=0x7f3a8400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3a997fa640 1 -- 172.21.15.122:0/322907007 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84005cf0 con 0x7f3a9c0b3de0 2024-03-16T20:45:10.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3a997fa640 1 -- 172.21.15.122:0/322907007 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a84005e50 con 0x7f3a9c0b3de0 2024-03-16T20:45:10.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3aa2521640 1 -- 172.21.15.122:0/322907007 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c08f000 con 0x7f3a9c0b3de0 2024-03-16T20:45:10.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2aa60cb640 1 Processor -- start 2024-03-16T20:45:10.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2aa60cb640 1 -- start start 2024-03-16T20:45:10.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2aa60cb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aa00b4a40 0x7f2aa0096b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2aa60cb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2aa00b7ce0 0x7f2aa01ec760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2aa60cb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2aa00b80c0 0x7f2aa01ecca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2aa60cb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa00985d0 con 0x7f2aa00b80c0 2024-03-16T20:45:10.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2aa60cb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa0099230 con 0x7f2aa00b4a40 2024-03-16T20:45:10.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2aa60cb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa0097fc0 con 0x7f2aa00b7ce0 2024-03-16T20:45:10.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f5bd7c66640 1 Processor -- start 2024-03-16T20:45:10.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3aa2521640 1 -- 172.21.15.122:0/322907007 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c14f000 con 0x7f3a9c0b3de0 2024-03-16T20:45:10.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f5bd7c66640 1 -- start start 2024-03-16T20:45:10.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3a997fa640 1 -- 172.21.15.122:0/322907007 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a840072d0 con 0x7f3a9c0b3de0 2024-03-16T20:45:10.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2a9effd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2aa00b7ce0 0x7f2aa01ec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2a9ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2aa00b80c0 0x7f2aa01ecca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f3aa2521640 1 -- 172.21.15.122:0/322907007 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9c14f240 con 0x7f3a9c0b3de0 2024-03-16T20:45:10.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2a9effd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2aa00b7ce0 0x7f2aa01ec760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:34306/0 (socket says 172.21.15.122:34306) 2024-03-16T20:45:10.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2a9effd640 1 -- 172.21.15.122:0/584218578 learned_addr learned my addr 172.21.15.122:0/584218578 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:10.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f026a16c640 1 --2- 172.21.15.122:0/2266863632 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0248066e40 0x7f0248069300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f02600ae000 tx=0x7f0260004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2a9ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2aa00b80c0 0x7f2aa01ecca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41474/0 (socket says 172.21.15.122:41474) 2024-03-16T20:45:10.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f5bd7c66640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bd0072220 0x7f5bd0141260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f5bd7c66640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bd00b6790 0x7f5bd014e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f5bd7c66640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bd0148250 0x7f5bd014e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f5bd7c66640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd0149970 con 0x7f5bd0072220 2024-03-16T20:45:10.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f5bd7c66640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd0148dd0 con 0x7f5bd00b6790 2024-03-16T20:45:10.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f5bd7c66640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd014a630 con 0x7f5bd0148250 2024-03-16T20:45:10.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2a9effd640 1 -- 172.21.15.122:0/584218578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aa00b4a40 msgr2=0x7f2aa0096b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2a9effd640 1 --2- 172.21.15.122:0/584218578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aa00b4a40 0x7f2aa0096b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f5bd6203640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bd0148250 0x7f5bd014e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f5bd5201640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bd00b6790 0x7f5bd014e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f5bd6203640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bd0148250 0x7f5bd014e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:34316/0 (socket says 172.21.15.122:34316) 2024-03-16T20:45:10.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f5bd6203640 1 -- 172.21.15.122:0/3973680211 learned_addr learned my addr 172.21.15.122:0/3973680211 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:10.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2a9effd640 1 -- 172.21.15.122:0/584218578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2aa00b80c0 msgr2=0x7f2aa01ecca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2a9effd640 1 --2- 172.21.15.122:0/584218578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2aa00b80c0 0x7f2aa01ecca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2a9effd640 1 -- 172.21.15.122:0/584218578 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9000ae50 con 0x7f2aa00b7ce0 2024-03-16T20:45:10.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f5bd5a02640 1 --2- 172.21.15.122:0/3973680211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bd0072220 0x7f5bd0141260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2a9ffff640 1 --2- 172.21.15.122:0/584218578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2aa00b80c0 0x7f2aa01ecca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:45:10.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f2a9effd640 1 --2- 172.21.15.122:0/584218578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2aa00b7ce0 0x7f2aa01ec760 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2a9401a000 tx=0x7f2a94019170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f5bd5a02640 1 -- 172.21.15.122:0/3973680211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bd0148250 msgr2=0x7f5bd014e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f5bd5a02640 1 --2- 172.21.15.122:0/3973680211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bd0148250 0x7f5bd014e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f5bd5a02640 1 -- 172.21.15.122:0/3973680211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bd00b6790 msgr2=0x7f5bd014e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f5bd5a02640 1 --2- 172.21.15.122:0/3973680211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bd00b6790 0x7f5bd014e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f5bd5a02640 1 -- 172.21.15.122:0/3973680211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc0094d0 con 0x7f5bd0072220 2024-03-16T20:45:10.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f5bd5201640 1 --2- 172.21.15.122:0/3973680211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bd00b6790 0x7f5bd014e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:45:10.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f5bd5a02640 1 --2- 172.21.15.122:0/3973680211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bd0072220 0x7f5bd0141260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000c410 tx=0x7f5bc000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f3a997fa640 1 -- 172.21.15.122:0/322907007 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f3a84007430 con 0x7f3a9c0b3de0 2024-03-16T20:45:10.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f3a997fa640 1 --2- 172.21.15.122:0/322907007 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3a7c066e70 0x7f3a7c069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f51c4594640 1 -- 172.21.15.122:0/1881626051 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a40052a0 con 0x7f51bc0b6a10 2024-03-16T20:45:10.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.236+0000 7f51b2ffd640 1 -- 172.21.15.122:0/1881626051 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f51ac003980 con 0x7f51bc0b6a10 2024-03-16T20:45:10.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f51b2ffd640 1 --2- 172.21.15.122:0/1881626051 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f51a0066f40 0x7f51a0069400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.282 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f51b2ffd640 1 -- 172.21.15.122:0/1881626051 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51ac0844c0 con 0x7f51bc0b6a10 2024-03-16T20:45:10.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f51c2330640 1 --2- 172.21.15.122:0/1881626051 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f51a0066f40 0x7f51a0069400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f3a997fa640 1 -- 172.21.15.122:0/322907007 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a840841d0 con 0x7f3a9c0b3de0 2024-03-16T20:45:10.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f51c2330640 1 --2- 172.21.15.122:0/1881626051 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f51a0066f40 0x7f51a0069400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f51b40af000 tx=0x7f51b40a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f3a93fff640 1 --2- 172.21.15.122:0/322907007 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3a7c066e70 0x7f3a7c069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7fcd277fe640 1 -- 172.21.15.122:0/2621859215 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3000b6c0 con 0x7fcd40154990 2024-03-16T20:45:10.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7fcd4540c640 1 -- 172.21.15.122:0/2621859215 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd400b2030 con 0x7fcd40154990 2024-03-16T20:45:10.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7fcd4540c640 1 -- 172.21.15.122:0/2621859215 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd40144e00 con 0x7fcd40154990 2024-03-16T20:45:10.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f5bc67fc640 1 -- 172.21.15.122:0/3973680211 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00069e0 con 0x7f5bd0072220 2024-03-16T20:45:10.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f12a97fa640 1 -- 172.21.15.122:0/1132671889 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a00b7820 con 0x7f12ac0b89f0 2024-03-16T20:45:10.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f12b1e4e640 1 -- 172.21.15.122:0/1132671889 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ac1ef530 con 0x7f12ac0b89f0 2024-03-16T20:45:10.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f12b1e4e640 1 -- 172.21.15.122:0/1132671889 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ac1ebfc0 con 0x7f12ac0b89f0 2024-03-16T20:45:10.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f12a97fa640 1 -- 172.21.15.122:0/1132671889 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a00b7980 con 0x7f12ac0b89f0 2024-03-16T20:45:10.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f3a93fff640 1 --2- 172.21.15.122:0/322907007 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3a7c066e70 0x7f3a7c069330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c14fd90 tx=0x7f3a8c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7fcd277fe640 1 -- 172.21.15.122:0/2621859215 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3000b820 con 0x7fcd40154990 2024-03-16T20:45:10.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7fcd277fe640 1 -- 172.21.15.122:0/2621859215 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3000bb90 con 0x7fcd40154990 2024-03-16T20:45:10.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f12a97fa640 1 -- 172.21.15.122:0/1132671889 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a00b7cf0 con 0x7f12ac0b89f0 2024-03-16T20:45:10.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f959f7fe640 1 -- 172.21.15.122:0/209578088 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac007a30 con 0x7f95b8072220 2024-03-16T20:45:10.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f959f7fe640 1 -- 172.21.15.122:0/209578088 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ac007b90 con 0x7f95b8072220 2024-03-16T20:45:10.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f959f7fe640 1 -- 172.21.15.122:0/209578088 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ac00ce70 con 0x7f95b8072220 2024-03-16T20:45:10.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f12b1e4e640 1 -- 172.21.15.122:0/1132671889 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128c0052a0 con 0x7f12ac0b89f0 2024-03-16T20:45:10.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f5bd7c66640 1 -- 172.21.15.122:0/3973680211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd0148b10 con 0x7f5bd0072220 2024-03-16T20:45:10.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f5bd7c66640 1 -- 172.21.15.122:0/3973680211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd0149150 con 0x7f5bd0072220 2024-03-16T20:45:10.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f5bd7c66640 1 -- 172.21.15.122:0/3973680211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd0149340 con 0x7f5bd0072220 2024-03-16T20:45:10.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f95bd00a640 1 -- 172.21.15.122:0/209578088 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b81efb10 con 0x7f95b8072220 2024-03-16T20:45:10.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f95bd00a640 1 -- 172.21.15.122:0/209578088 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b81de980 con 0x7f95b8072220 2024-03-16T20:45:10.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7fcd4540c640 1 -- 172.21.15.122:0/2621859215 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd200052a0 con 0x7fcd40154990 2024-03-16T20:45:10.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.240+0000 7f025affd640 1 -- 172.21.15.122:0/2266863632 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f025c054510 con 0x7f02640aed50 2024-03-16T20:45:10.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.244+0000 7f95bd00a640 1 -- 172.21.15.122:0/209578088 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b80684a0 con 0x7f95b8072220 2024-03-16T20:45:10.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.244+0000 7fcd277fe640 1 -- 172.21.15.122:0/2621859215 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fcd3000bcf0 con 0x7fcd40154990 2024-03-16T20:45:10.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.244+0000 7fcd277fe640 1 --2- 172.21.15.122:0/2621859215 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcd2c066e40 0x7fcd2c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.244+0000 7fcd3effd640 1 --2- 172.21.15.122:0/2621859215 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcd2c066e40 0x7fcd2c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.244+0000 7f959f7fe640 1 -- 172.21.15.122:0/209578088 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f95ac0113f0 con 0x7f95b8072220 2024-03-16T20:45:10.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.244+0000 7f959f7fe640 1 --2- 172.21.15.122:0/209578088 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f95a4066e90 0x7f95a4069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.244+0000 7f95b659c640 1 --2- 172.21.15.122:0/209578088 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f95a4066e90 0x7f95a4069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.244+0000 7fcd3effd640 1 --2- 172.21.15.122:0/2621859215 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcd2c066e40 0x7fcd2c069300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcd380012f0 tx=0x7fcd38007c20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.244+0000 7f95b659c640 1 --2- 172.21.15.122:0/209578088 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f95a4066e90 0x7f95a4069350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f95b00b0000 tx=0x7f95b00a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.244+0000 7fcd277fe640 1 -- 172.21.15.122:0/2621859215 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd30056cf0 con 0x7fcd40154990 2024-03-16T20:45:10.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.244+0000 7f5bc67fc640 1 -- 172.21.15.122:0/3973680211 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc0006b40 con 0x7f5bd0072220 2024-03-16T20:45:10.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.244+0000 7f12a97fa640 1 -- 172.21.15.122:0/1132671889 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f12a00ba480 con 0x7f12ac0b89f0 2024-03-16T20:45:10.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.244+0000 7f12a97fa640 1 --2- 172.21.15.122:0/1132671889 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1290066e40 0x7f1290069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.244+0000 7f12a97fa640 1 -- 172.21.15.122:0/1132671889 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12a00f0dc0 con 0x7f12ac0b89f0 2024-03-16T20:45:10.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.244+0000 7f12b0e4c640 1 --2- 172.21.15.122:0/1132671889 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1290066e40 0x7f1290069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.244+0000 7f959f7fe640 1 -- 172.21.15.122:0/209578088 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95ac055bd0 con 0x7f95b8072220 2024-03-16T20:45:10.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.248+0000 7f5bc67fc640 1 -- 172.21.15.122:0/3973680211 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0007030 con 0x7f5bd0072220 2024-03-16T20:45:10.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.248+0000 7f12b0e4c640 1 --2- 172.21.15.122:0/1132671889 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1290066e40 0x7f1290069300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1298000f70 tx=0x7f12980079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.292 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.252+0000 7fcd277fe640 1 -- 172.21.15.122:0/2621859215 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fcd30052890 con 0x7fcd40154990 2024-03-16T20:45:10.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.252+0000 7f2a87fff640 1 -- 172.21.15.122:0/584218578 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94004500 con 0x7f2aa00b7ce0 2024-03-16T20:45:10.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.252+0000 7f2a87fff640 1 -- 172.21.15.122:0/584218578 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a94003620 con 0x7f2aa00b7ce0 2024-03-16T20:45:10.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.252+0000 7f2a87fff640 1 -- 172.21.15.122:0/584218578 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a940039d0 con 0x7f2aa00b7ce0 2024-03-16T20:45:10.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.252+0000 7f2aa60cb640 1 -- 172.21.15.122:0/584218578 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a940096f0 con 0x7f2aa00b7ce0 2024-03-16T20:45:10.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.252+0000 7f2aa60cb640 1 -- 172.21.15.122:0/584218578 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa0099480 con 0x7f2aa00b7ce0 2024-03-16T20:45:10.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.252+0000 7f51b2ffd640 1 -- 172.21.15.122:0/1881626051 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f51ac0513d0 con 0x7f51bc0b6a10 2024-03-16T20:45:10.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.252+0000 7f3a997fa640 1 -- 172.21.15.122:0/322907007 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3a840510e0 con 0x7f3a9c0b3de0 2024-03-16T20:45:10.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.252+0000 7f2aa60cb640 1 -- 172.21.15.122:0/584218578 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa00b3b50 con 0x7f2aa00b7ce0 2024-03-16T20:45:10.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.256+0000 7f2a87fff640 1 -- 172.21.15.122:0/584218578 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f2a94003b30 con 0x7f2aa00b7ce0 2024-03-16T20:45:10.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.256+0000 7f2a87fff640 1 --2- 172.21.15.122:0/584218578 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2a8c066e90 0x7f2a8c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.256+0000 7f2a87fff640 1 -- 172.21.15.122:0/584218578 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a940a8050 con 0x7f2aa00b7ce0 2024-03-16T20:45:10.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.260+0000 7f2a9f7fe640 1 --2- 172.21.15.122:0/584218578 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2a8c066e90 0x7f2a8c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.260+0000 7f2a9f7fe640 1 --2- 172.21.15.122:0/584218578 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2a8c066e90 0x7f2a8c069350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800ab90 tx=0x7f2a98009230 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.260+0000 7f959f7fe640 1 -- 172.21.15.122:0/209578088 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f95ac051e70 con 0x7f95b8072220 2024-03-16T20:45:10.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.268+0000 7f2a87fff640 1 -- 172.21.15.122:0/584218578 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2a940f4b50 con 0x7f2aa00b7ce0 2024-03-16T20:45:10.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.268+0000 7f12a97fa640 1 -- 172.21.15.122:0/1132671889 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f12a00ec960 con 0x7f12ac0b89f0 2024-03-16T20:45:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.272+0000 7f5bc67fc640 1 -- 172.21.15.122:0/3973680211 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f5bc0007190 con 0x7f5bd0072220 2024-03-16T20:45:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.272+0000 7f5bc67fc640 1 --2- 172.21.15.122:0/3973680211 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5bb4066e40 0x7f5bb4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.272+0000 7f5bd5201640 1 --2- 172.21.15.122:0/3973680211 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5bb4066e40 0x7f5bb4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.272+0000 7f5bc67fc640 1 -- 172.21.15.122:0/3973680211 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bc0054f70 con 0x7f5bd0072220 2024-03-16T20:45:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.272+0000 7f5bd5201640 1 --2- 172.21.15.122:0/3973680211 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5bb4066e40 0x7f5bb4069300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0b1000 tx=0x7f5bcc0a9000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:10.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.280+0000 7f5bc67fc640 1 -- 172.21.15.122:0/3973680211 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5bc0051210 con 0x7f5bd0072220 2024-03-16T20:45:10.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.552+0000 7f026b16e640 1 -- 172.21.15.122:0/2266863632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f02640726f0 con 0x7f02640aed50 2024-03-16T20:45:10.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.552+0000 7f025affd640 1 -- 172.21.15.122:0/2266863632 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f025c023090 con 0x7f02640aed50 2024-03-16T20:45:10.562 INFO:teuthology.orchestra.run.smithi122.stdout:55834574853 2024-03-16T20:45:10.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.556+0000 7f0258ff9640 1 -- 172.21.15.122:0/2266863632 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0248066e40 msgr2=0x7f0248069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.556+0000 7f0258ff9640 1 --2- 172.21.15.122:0/2266863632 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0248066e40 0x7f0248069300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f02600ae000 tx=0x7f0260004740 comp rx=0 tx=0).stop 2024-03-16T20:45:10.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.556+0000 7f0258ff9640 1 -- 172.21.15.122:0/2266863632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02640aed50 msgr2=0x7f026413ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.556+0000 7f0258ff9640 1 --2- 172.21.15.122:0/2266863632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02640aed50 0x7f026413ea00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f025c00e450 tx=0x7f025c00e910 comp rx=0 tx=0).stop 2024-03-16T20:45:10.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.560+0000 7f0258ff9640 1 -- 172.21.15.122:0/2266863632 shutdown_connections 2024-03-16T20:45:10.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.560+0000 7f0258ff9640 1 --2- 172.21.15.122:0/2266863632 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0248066e40 0x7f0248069300 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.560+0000 7f0258ff9640 1 --2- 172.21.15.122:0/2266863632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f026413ef40 0x7f0264071590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.560+0000 7f0258ff9640 1 --2- 172.21.15.122:0/2266863632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02640aed50 0x7f026413ea00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.560+0000 7f0258ff9640 1 --2- 172.21.15.122:0/2266863632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f026406d580 0x7f0264143790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.560+0000 7f0258ff9640 1 -- 172.21.15.122:0/2266863632 >> 172.21.15.122:0/2266863632 conn(0x7f026409a350 msgr2=0x7f02640b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.560+0000 7f0258ff9640 1 -- 172.21.15.122:0/2266863632 shutdown_connections 2024-03-16T20:45:10.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.560+0000 7f0258ff9640 1 -- 172.21.15.122:0/2266863632 wait complete. 2024-03-16T20:45:10.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.568+0000 7fcd4540c640 1 -- 172.21.15.122:0/2621859215 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcd20005820 con 0x7fcd40154990 2024-03-16T20:45:10.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.568+0000 7fcd277fe640 1 -- 172.21.15.122:0/2621859215 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcd30055f50 con 0x7fcd40154990 2024-03-16T20:45:10.578 INFO:teuthology.orchestra.run.smithi122.stdout:55834574854 2024-03-16T20:45:10.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.572+0000 7fcd257fa640 1 -- 172.21.15.122:0/2621859215 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcd2c066e40 msgr2=0x7fcd2c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.572+0000 7fcd257fa640 1 --2- 172.21.15.122:0/2621859215 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcd2c066e40 0x7fcd2c069300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcd380012f0 tx=0x7fcd38007c20 comp rx=0 tx=0).stop 2024-03-16T20:45:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.572+0000 7fcd257fa640 1 -- 172.21.15.122:0/2621859215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd40154990 msgr2=0x7fcd40154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.572+0000 7fcd257fa640 1 --2- 172.21.15.122:0/2621859215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd40154990 0x7fcd40154e10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000d800 tx=0x7fcd3000dcc0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.572+0000 7fcd257fa640 1 -- 172.21.15.122:0/2621859215 shutdown_connections 2024-03-16T20:45:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.572+0000 7fcd257fa640 1 --2- 172.21.15.122:0/2621859215 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcd2c066e40 0x7fcd2c069300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.572+0000 7fcd257fa640 1 --2- 172.21.15.122:0/2621859215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcd40154990 0x7fcd40154e10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.572+0000 7fcd257fa640 1 --2- 172.21.15.122:0/2621859215 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcd401453b0 0x7fcd400b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.572+0000 7fcd257fa640 1 --2- 172.21.15.122:0/2621859215 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcd400b6390 0x7fcd400b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.572+0000 7fcd257fa640 1 -- 172.21.15.122:0/2621859215 >> 172.21.15.122:0/2621859215 conn(0x7fcd4006c0a0 msgr2=0x7fcd40074320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.572+0000 7fcd257fa640 1 -- 172.21.15.122:0/2621859215 shutdown_connections 2024-03-16T20:45:10.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.572+0000 7fcd257fa640 1 -- 172.21.15.122:0/2621859215 wait complete. 2024-03-16T20:45:10.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.572+0000 7f3aa2521640 1 -- 172.21.15.122:0/322907007 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3a9c14f240 con 0x7f3a9c0b3de0 2024-03-16T20:45:10.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.572+0000 7f51c4594640 1 -- 172.21.15.122:0/1881626051 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f51a4005820 con 0x7f51bc0b6a10 2024-03-16T20:45:10.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.572+0000 7f3a997fa640 1 -- 172.21.15.122:0/322907007 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a9c14f240 con 0x7f3a9c0b3de0 2024-03-16T20:45:10.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.576+0000 7f51b2ffd640 1 -- 172.21.15.122:0/1881626051 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f51ac017020 con 0x7f51bc0b6a10 2024-03-16T20:45:10.583 INFO:teuthology.orchestra.run.smithi122.stdout:55834574853 2024-03-16T20:45:10.583 INFO:teuthology.orchestra.run.smithi122.stdout:55834574853 2024-03-16T20:45:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f51c4594640 1 -- 172.21.15.122:0/1881626051 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f51a0066f40 msgr2=0x7f51a0069400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f3aa2521640 1 -- 172.21.15.122:0/322907007 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3a7c066e70 msgr2=0x7f3a7c069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f3aa2521640 1 --2- 172.21.15.122:0/322907007 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3a7c066e70 0x7f3a7c069330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c14fd90 tx=0x7f3a8c008040 comp rx=0 tx=0).stop 2024-03-16T20:45:10.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f51c4594640 1 --2- 172.21.15.122:0/1881626051 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f51a0066f40 0x7f51a0069400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f51b40af000 tx=0x7f51b40a7070 comp rx=0 tx=0).stop 2024-03-16T20:45:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f51c4594640 1 -- 172.21.15.122:0/1881626051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51bc0b6a10 msgr2=0x7f51bc0b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f51c4594640 1 --2- 172.21.15.122:0/1881626051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51bc0b6a10 0x7f51bc0b1c70 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00c390 tx=0x7f51ac00c850 comp rx=0 tx=0).stop 2024-03-16T20:45:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f51c4594640 1 -- 172.21.15.122:0/1881626051 shutdown_connections 2024-03-16T20:45:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f3aa2521640 1 -- 172.21.15.122:0/322907007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a9c0b3de0 msgr2=0x7f3a9c0af550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f51c4594640 1 --2- 172.21.15.122:0/1881626051 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f51a0066f40 0x7f51a0069400 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f3aa2521640 1 --2- 172.21.15.122:0/322907007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a9c0b3de0 0x7f3a9c0af550 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3a8400c430 tx=0x7f3a8400c8f0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f51c4594640 1 --2- 172.21.15.122:0/1881626051 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51bc0b7600 0x7f51bc1e46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f51c4594640 1 --2- 172.21.15.122:0/1881626051 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51bc0b6a10 0x7f51bc0b1c70 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f51c4594640 1 --2- 172.21.15.122:0/1881626051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51bc0b5610 0x7f51bc0b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f51c4594640 1 -- 172.21.15.122:0/1881626051 >> 172.21.15.122:0/1881626051 conn(0x7f51bc09de70 msgr2=0x7f51bc148240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f51c4594640 1 -- 172.21.15.122:0/1881626051 shutdown_connections 2024-03-16T20:45:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f51c4594640 1 -- 172.21.15.122:0/1881626051 wait complete. 2024-03-16T20:45:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f3aa2521640 1 -- 172.21.15.122:0/322907007 shutdown_connections 2024-03-16T20:45:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f3aa2521640 1 --2- 172.21.15.122:0/322907007 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3a7c066e70 0x7f3a7c069330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f3aa2521640 1 --2- 172.21.15.122:0/322907007 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a9c14f430 0x7f3a9c08d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f3aa2521640 1 --2- 172.21.15.122:0/322907007 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a9c0b8550 0x7f3a9c08d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f3aa2521640 1 --2- 172.21.15.122:0/322907007 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a9c0b3de0 0x7f3a9c0af550 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f3aa2521640 1 -- 172.21.15.122:0/322907007 >> 172.21.15.122:0/322907007 conn(0x7f3a9c06de30 msgr2=0x7f3a9c09d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f3aa2521640 1 -- 172.21.15.122:0/322907007 shutdown_connections 2024-03-16T20:45:10.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.580+0000 7f3aa2521640 1 -- 172.21.15.122:0/322907007 wait complete. 2024-03-16T20:45:10.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-03-16T20:45:10.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.584+0000 7f95bd00a640 1 -- 172.21.15.122:0/209578088 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f95b81efd00 con 0x7f95b8072220 2024-03-16T20:45:10.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.584+0000 7f959f7fe640 1 -- 172.21.15.122:0/209578088 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f95ac055530 con 0x7f95b8072220 2024-03-16T20:45:10.591 INFO:teuthology.orchestra.run.smithi122.stdout:55834574854 2024-03-16T20:45:10.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.588+0000 7f95bd00a640 1 -- 172.21.15.122:0/209578088 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f95a4066e90 msgr2=0x7f95a4069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.588+0000 7f95bd00a640 1 --2- 172.21.15.122:0/209578088 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f95a4066e90 0x7f95a4069350 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f95b00b0000 tx=0x7f95b00a9070 comp rx=0 tx=0).stop 2024-03-16T20:45:10.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.588+0000 7f95bd00a640 1 -- 172.21.15.122:0/209578088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8072220 msgr2=0x7f95b81e2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.588+0000 7f95bd00a640 1 --2- 172.21.15.122:0/209578088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8072220 0x7f95b81e2420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00c7c0 tx=0x7f95ac00cc80 comp rx=0 tx=0).stop 2024-03-16T20:45:10.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.588+0000 7f12b1e4e640 1 -- 172.21.15.122:0/1132671889 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f128c005820 con 0x7f12ac0b89f0 2024-03-16T20:45:10.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.588+0000 7f95bd00a640 1 -- 172.21.15.122:0/209578088 shutdown_connections 2024-03-16T20:45:10.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.588+0000 7f95bd00a640 1 --2- 172.21.15.122:0/209578088 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f95a4066e90 0x7f95a4069350 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.588+0000 7f95bd00a640 1 --2- 172.21.15.122:0/209578088 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f95b81f0570 0x7f95b81dd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.588+0000 7f95bd00a640 1 --2- 172.21.15.122:0/209578088 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f95b80b6790 0x7f95b81f0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.588+0000 7f95bd00a640 1 --2- 172.21.15.122:0/209578088 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f95b8072220 0x7f95b81e2420 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.588+0000 7f95bd00a640 1 -- 172.21.15.122:0/209578088 >> 172.21.15.122:0/209578088 conn(0x7f95b806c0a0 msgr2=0x7f95b806b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.588+0000 7f12a97fa640 1 -- 172.21.15.122:0/1132671889 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f12a00ab070 con 0x7f12ac0b89f0 2024-03-16T20:45:10.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.588+0000 7f95bd00a640 1 -- 172.21.15.122:0/209578088 shutdown_connections 2024-03-16T20:45:10.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.588+0000 7f95bd00a640 1 -- 172.21.15.122:0/209578088 wait complete. 2024-03-16T20:45:10.597 INFO:teuthology.orchestra.run.smithi122.stdout:55834574853 2024-03-16T20:45:10.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.588+0000 7f2aa60cb640 1 -- 172.21.15.122:0/584218578 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2aa0074bb0 con 0x7f2aa00b7ce0 2024-03-16T20:45:10.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.588+0000 7f2a87fff640 1 -- 172.21.15.122:0/584218578 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2a940b0090 con 0x7f2aa00b7ce0 2024-03-16T20:45:10.597 INFO:teuthology.orchestra.run.smithi122.stdout:55834574853 2024-03-16T20:45:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f12b1e4e640 1 -- 172.21.15.122:0/1132671889 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1290066e40 msgr2=0x7f1290069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f12b1e4e640 1 --2- 172.21.15.122:0/1132671889 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1290066e40 0x7f1290069300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1298000f70 tx=0x7f12980079c0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f12b1e4e640 1 -- 172.21.15.122:0/1132671889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12ac0b89f0 msgr2=0x7f12ac1efd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f12b1e4e640 1 --2- 172.21.15.122:0/1132671889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12ac0b89f0 0x7f12ac1efd30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f12a001af60 tx=0x7f12a0019650 comp rx=0 tx=0).stop 2024-03-16T20:45:10.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f12b1e4e640 1 -- 172.21.15.122:0/1132671889 shutdown_connections 2024-03-16T20:45:10.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f12b1e4e640 1 --2- 172.21.15.122:0/1132671889 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1290066e40 0x7f1290069300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f12b1e4e640 1 --2- 172.21.15.122:0/1132671889 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f12ac1434f0 0x7f12ac1ec3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f12b1e4e640 1 --2- 172.21.15.122:0/1132671889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f12ac0b89f0 0x7f12ac1efd30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f12b1e4e640 1 --2- 172.21.15.122:0/1132671889 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f12ac072f20 0x7f12ac1e2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f12b1e4e640 1 -- 172.21.15.122:0/1132671889 >> 172.21.15.122:0/1132671889 conn(0x7f12ac099c20 msgr2=0x7f12ac09aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f12b1e4e640 1 -- 172.21.15.122:0/1132671889 shutdown_connections 2024-03-16T20:45:10.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f12b1e4e640 1 -- 172.21.15.122:0/1132671889 wait complete. 2024-03-16T20:45:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f2aa60cb640 1 -- 172.21.15.122:0/584218578 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2a8c066e90 msgr2=0x7f2a8c069350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f2aa60cb640 1 --2- 172.21.15.122:0/584218578 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2a8c066e90 0x7f2a8c069350 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2a9800ab90 tx=0x7f2a98009230 comp rx=0 tx=0).stop 2024-03-16T20:45:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f2aa60cb640 1 -- 172.21.15.122:0/584218578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2aa00b7ce0 msgr2=0x7f2aa01ec760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f2aa60cb640 1 --2- 172.21.15.122:0/584218578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2aa00b7ce0 0x7f2aa01ec760 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2a9401a000 tx=0x7f2a94019170 comp rx=0 tx=0).stop 2024-03-16T20:45:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f2aa60cb640 1 -- 172.21.15.122:0/584218578 shutdown_connections 2024-03-16T20:45:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f2aa60cb640 1 --2- 172.21.15.122:0/584218578 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2a8c066e90 0x7f2a8c069350 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f2aa60cb640 1 --2- 172.21.15.122:0/584218578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2aa00b80c0 0x7f2aa01ecca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f2aa60cb640 1 --2- 172.21.15.122:0/584218578 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2aa00b7ce0 0x7f2aa01ec760 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f2aa60cb640 1 --2- 172.21.15.122:0/584218578 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2aa00b4a40 0x7f2aa0096b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f2aa60cb640 1 -- 172.21.15.122:0/584218578 >> 172.21.15.122:0/584218578 conn(0x7f2aa009a4c0 msgr2=0x7f2aa0155210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f2aa60cb640 1 -- 172.21.15.122:0/584218578 shutdown_connections 2024-03-16T20:45:10.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.592+0000 7f2aa60cb640 1 -- 172.21.15.122:0/584218578 wait complete. 2024-03-16T20:45:10.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.596+0000 7f5bd7c66640 1 -- 172.21.15.122:0/3973680211 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f5bd0074bb0 con 0x7f5bd0072220 2024-03-16T20:45:10.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.596+0000 7f5bc67fc640 1 -- 172.21.15.122:0/3973680211 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5bc0017020 con 0x7f5bd0072220 2024-03-16T20:45:10.604 INFO:teuthology.orchestra.run.smithi122.stdout:55834574853 2024-03-16T20:45:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.600+0000 7f5bd7c66640 1 -- 172.21.15.122:0/3973680211 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5bb4066e40 msgr2=0x7f5bb4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.600+0000 7f5bd7c66640 1 --2- 172.21.15.122:0/3973680211 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5bb4066e40 0x7f5bb4069300 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0b1000 tx=0x7f5bcc0a9000 comp rx=0 tx=0).stop 2024-03-16T20:45:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.600+0000 7f5bd7c66640 1 -- 172.21.15.122:0/3973680211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bd0072220 msgr2=0x7f5bd0141260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.600+0000 7f5bd7c66640 1 --2- 172.21.15.122:0/3973680211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bd0072220 0x7f5bd0141260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000c410 tx=0x7f5bc000c8d0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.600+0000 7f5bd7c66640 1 -- 172.21.15.122:0/3973680211 shutdown_connections 2024-03-16T20:45:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.600+0000 7f5bd7c66640 1 --2- 172.21.15.122:0/3973680211 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5bb4066e40 0x7f5bb4069300 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.600+0000 7f5bd7c66640 1 --2- 172.21.15.122:0/3973680211 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5bd0148250 0x7f5bd014e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.600+0000 7f5bd7c66640 1 --2- 172.21.15.122:0/3973680211 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5bd00b6790 0x7f5bd014e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.600+0000 7f5bd7c66640 1 --2- 172.21.15.122:0/3973680211 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5bd0072220 0x7f5bd0141260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:10.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.600+0000 7f5bd7c66640 1 -- 172.21.15.122:0/3973680211 >> 172.21.15.122:0/3973680211 conn(0x7f5bd006c0a0 msgr2=0x7f5bd00b9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:10.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.600+0000 7f5bd7c66640 1 -- 172.21.15.122:0/3973680211 shutdown_connections 2024-03-16T20:45:10.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:10.600+0000 7f5bd7c66640 1 -- 172.21.15.122:0/3973680211 wait complete. 2024-03-16T20:45:10.609 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2024-03-16T20:45:10.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-03-16T20:45:10.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-03-16T20:45:10.626 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2024-03-16T20:45:10.628 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2024-03-16T20:45:10.635 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2024-03-16T20:45:10.637 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2024-03-16T20:45:11.592 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-03-16T20:45:11.611 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-03-16T20:45:11.618 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-03-16T20:45:11.623 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-03-16T20:45:11.627 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-03-16T20:45:11.629 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-03-16T20:45:11.635 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-03-16T20:45:11.637 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-03-16T20:45:11.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.772+0000 7f9da3bfe640 1 Processor -- start 2024-03-16T20:45:11.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.772+0000 7f9da3bfe640 1 -- start start 2024-03-16T20:45:11.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.772+0000 7f9da3bfe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9d9c0b5190 0x7f9d9c1435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.772+0000 7f9da3bfe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d9c0b5570 0x7f9d9c1403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.772+0000 7f9da3bfe640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d9c0b59f0 con 0x7f9d9c140930 2024-03-16T20:45:11.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.772+0000 7f9da3bfe640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d9c143c50 con 0x7f9d9c0b5190 2024-03-16T20:45:11.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.772+0000 7f9da3bfe640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c143f30 con 0x7f9d9c0b5570 2024-03-16T20:45:11.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.772+0000 7f9da199a640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9d9c0b5190 0x7f9d9c1435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.772+0000 7f9da199a640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9d9c0b5190 0x7f9d9c1435e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41502/0 (socket says 172.21.15.122:41502) 2024-03-16T20:45:11.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.772+0000 7f9da199a640 1 -- 172.21.15.122:0/365665009 learned_addr learned my addr 172.21.15.122:0/365665009 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:11.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.772+0000 7f9da199a640 1 -- 172.21.15.122:0/365665009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d9c0b5570 msgr2=0x7f9d9c1403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:11.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.772+0000 7f9da199a640 1 --2- 172.21.15.122:0/365665009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d9c0b5570 0x7f9d9c1403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.772+0000 7f9da199a640 1 -- 172.21.15.122:0/365665009 >> v1:172.21.15.122:6789/0 conn(0x7f9d9c140930 legacy=0x7f9d9c06ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.772+0000 7f9da199a640 1 -- 172.21.15.122:0/365665009 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9c06c970 con 0x7f9d9c0b5190 2024-03-16T20:45:11.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.772+0000 7f9da0998640 1 -- 172.21.15.122:0/365665009 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 881527522 0 0) 0x7f9d9c0b59f0 con 0x7f9d9c140930 2024-03-16T20:45:11.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.776+0000 7f9da199a640 1 --2- 172.21.15.122:0/365665009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d9c0b5190 0x7f9d9c1435e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400c770 tx=0x7f9d9401d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d418bc85f7a50b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.776+0000 7f9da0998640 1 -- 172.21.15.122:0/365665009 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d940aa410 con 0x7f9d9c0b5190 2024-03-16T20:45:11.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.776+0000 7f9da0998640 1 -- 172.21.15.122:0/365665009 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d940aa570 con 0x7f9d9c0b5190 2024-03-16T20:45:11.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.776+0000 7f9da3bfe640 1 -- 172.21.15.122:0/365665009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d9c0b5190 msgr2=0x7f9d9c1435e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.776+0000 7f9da3bfe640 1 --2- 172.21.15.122:0/365665009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d9c0b5190 0x7f9d9c1435e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400c770 tx=0x7f9d9401d490 comp rx=0 tx=0).stop 2024-03-16T20:45:11.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da3bfe640 1 -- 172.21.15.122:0/365665009 shutdown_connections 2024-03-16T20:45:11.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da3bfe640 1 --2- 172.21.15.122:0/365665009 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d9c0b5570 0x7f9d9c1403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da3bfe640 1 --2- 172.21.15.122:0/365665009 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d9c0b5190 0x7f9d9c1435e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da3bfe640 1 -- 172.21.15.122:0/365665009 >> 172.21.15.122:0/365665009 conn(0x7f9d9c09de70 msgr2=0x7f9d9c0b86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:11.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da3bfe640 1 -- 172.21.15.122:0/365665009 shutdown_connections 2024-03-16T20:45:11.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da3bfe640 1 -- 172.21.15.122:0/365665009 wait complete. 2024-03-16T20:45:11.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da3bfe640 1 Processor -- start 2024-03-16T20:45:11.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da3bfe640 1 -- start start 2024-03-16T20:45:11.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da3bfe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d9c0b5190 0x7f9d9c137760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da3bfe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d9c0b5570 0x7f9d9c0716c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da3bfe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d9c140930 0x7f9d9c1522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da3bfe640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c13b570 con 0x7f9d9c140930 2024-03-16T20:45:11.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da3bfe640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c1e7ce0 con 0x7f9d9c0b5570 2024-03-16T20:45:11.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da3bfe640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9c1e8170 con 0x7f9d9c0b5190 2024-03-16T20:45:11.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da1199640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d9c0b5570 0x7f9d9c0716c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da219b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d9c140930 0x7f9d9c1522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da1199640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d9c0b5570 0x7f9d9c0716c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.122:45282/0 (socket says 172.21.15.122:45282) 2024-03-16T20:45:11.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da219b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d9c140930 0x7f9d9c1522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41510/0 (socket says 172.21.15.122:41510) 2024-03-16T20:45:11.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da1199640 1 -- 172.21.15.122:0/1447573112 learned_addr learned my addr 172.21.15.122:0/1447573112 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:11.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.780+0000 7f9da199a640 1 --2- 172.21.15.122:0/1447573112 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d9c0b5190 0x7f9d9c137760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.784+0000 7f9da1199640 1 -- 172.21.15.122:0/1447573112 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d9c0b5190 msgr2=0x7f9d9c137760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.784+0000 7f9da1199640 1 --2- 172.21.15.122:0/1447573112 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d9c0b5190 0x7f9d9c137760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.784+0000 7f9da1199640 1 -- 172.21.15.122:0/1447573112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d9c140930 msgr2=0x7f9d9c1522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.784+0000 7f9da1199640 1 --2- 172.21.15.122:0/1447573112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d9c140930 0x7f9d9c1522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.784+0000 7f9da1199640 1 -- 172.21.15.122:0/1447573112 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9400c3d0 con 0x7f9d9c0b5570 2024-03-16T20:45:11.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.784+0000 7f9da1199640 1 --2- 172.21.15.122:0/1447573112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d9c0b5570 0x7f9d9c0716c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00b550 tx=0x7f9d8c00ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.788+0000 7f9d927fc640 1 -- 172.21.15.122:0/1447573112 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c010870 con 0x7f9d9c0b5570 2024-03-16T20:45:11.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.788+0000 7f9da3bfe640 1 -- 172.21.15.122:0/1447573112 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d9c1e7690 con 0x7f9d9c0b5570 2024-03-16T20:45:11.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.788+0000 7f9da3bfe640 1 -- 172.21.15.122:0/1447573112 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9c073450 con 0x7f9d9c0b5570 2024-03-16T20:45:11.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.788+0000 7f9d927fc640 1 -- 172.21.15.122:0/1447573112 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8c0109f0 con 0x7f9d9c0b5570 2024-03-16T20:45:11.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.788+0000 7f9d927fc640 1 -- 172.21.15.122:0/1447573112 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c010da0 con 0x7f9d9c0b5570 2024-03-16T20:45:11.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.788+0000 7f9da3bfe640 1 -- 172.21.15.122:0/1447573112 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d840052a0 con 0x7f9d9c0b5570 2024-03-16T20:45:11.792 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.788+0000 7f9d927fc640 1 -- 172.21.15.122:0/1447573112 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f9d8c037580 con 0x7f9d9c0b5570 2024-03-16T20:45:11.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.788+0000 7f9d927fc640 1 --2- 172.21.15.122:0/1447573112 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9d80066ec0 0x7f9d80069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.788+0000 7f9d927fc640 1 -- 172.21.15.122:0/1447573112 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d8c084b80 con 0x7f9d9c0b5570 2024-03-16T20:45:11.793 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.788+0000 7f9da199a640 1 --2- 172.21.15.122:0/1447573112 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9d80066ec0 0x7f9d80069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.792+0000 7f9da199a640 1 --2- 172.21.15.122:0/1447573112 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9d80066ec0 0x7f9d80069380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d940ae000 tx=0x7f9d940a8000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.800+0000 7f9d927fc640 1 -- 172.21.15.122:0/1447573112 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9d8c051970 con 0x7f9d9c0b5570 2024-03-16T20:45:11.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.808+0000 7f0b41f14640 1 Processor -- start 2024-03-16T20:45:11.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.808+0000 7f0b41f14640 1 -- start start 2024-03-16T20:45:11.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.808+0000 7f0b41f14640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0b3c144290 0x7f0b3c06ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.808+0000 7f0b41f14640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0b3c0b90c0 0x7f0b3c0b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.808+0000 7f0b3b7fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0b3c144290 0x7f0b3c06ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.808+0000 7f0b3b7fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0b3c144290 0x7f0b3c06ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.122:45288/0 (socket says 172.21.15.122:45288) 2024-03-16T20:45:11.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.808+0000 7f0b3b7fe640 1 -- 172.21.15.122:0/783746117 learned_addr learned my addr 172.21.15.122:0/783746117 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:11.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.808+0000 7f0b41f14640 1 -- 172.21.15.122:0/783746117 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3c06daf0 con 0x7f0b3c144c00 2024-03-16T20:45:11.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.808+0000 7f0b41f14640 1 -- 172.21.15.122:0/783746117 --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c0b1390 con 0x7f0b3c0b90c0 2024-03-16T20:45:11.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.808+0000 7f0b41f14640 1 -- 172.21.15.122:0/783746117 --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b3c06dd40 con 0x7f0b3c144290 2024-03-16T20:45:11.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.808+0000 7f0b3affd640 1 --2- 172.21.15.122:0/783746117 >> v2:172.21.15.122:3300/0 conn(0x7f0b3c0b90c0 0x7f0b3c0b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.808+0000 7f0b3a7fc640 1 -- 172.21.15.122:0/783746117 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 68791759 0 0) 0x7f0b3c06daf0 con 0x7f0b3c144c00 2024-03-16T20:45:11.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.808+0000 7f0b3a7fc640 1 -- 172.21.15.122:0/783746117 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b24003530 con 0x7f0b3c144c00 2024-03-16T20:45:11.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.808+0000 7f0b3b7fe640 1 -- 172.21.15.122:0/783746117 >> v2:172.21.15.122:3300/0 conn(0x7f0b3c0b90c0 msgr2=0x7f0b3c0b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:11.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.808+0000 7f0b3b7fe640 1 --2- 172.21.15.122:0/783746117 >> v2:172.21.15.122:3300/0 conn(0x7f0b3c0b90c0 0x7f0b3c0b7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.808+0000 7f0b3b7fe640 1 -- 172.21.15.122:0/783746117 >> v1:172.21.15.122:6789/0 conn(0x7f0b3c144c00 legacy=0x7f0b3c13df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.808+0000 7f0b3b7fe640 1 -- 172.21.15.122:0/783746117 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c13e690 con 0x7f0b3c144290 2024-03-16T20:45:11.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.812+0000 7f0b3b7fe640 1 --2- 172.21.15.122:0/783746117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b3c144290 0x7f0b3c06ed70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c067d00 tx=0x7f0b3001c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=433d3008bf458786 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.812+0000 7f0b3a7fc640 1 -- 172.21.15.122:0/783746117 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b300ab620 con 0x7f0b3c144290 2024-03-16T20:45:11.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.812+0000 7f0b3a7fc640 1 -- 172.21.15.122:0/783746117 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b300ab780 con 0x7f0b3c144290 2024-03-16T20:45:11.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.812+0000 7f0b3a7fc640 1 -- 172.21.15.122:0/783746117 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b300abb10 con 0x7f0b3c144290 2024-03-16T20:45:11.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.812+0000 7f0b41f14640 1 -- 172.21.15.122:0/783746117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b3c144290 msgr2=0x7f0b3c06ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.812+0000 7f0b41f14640 1 --2- 172.21.15.122:0/783746117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b3c144290 0x7f0b3c06ed70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c067d00 tx=0x7f0b3001c400 comp rx=0 tx=0).stop 2024-03-16T20:45:11.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.812+0000 7f0b41f14640 1 -- 172.21.15.122:0/783746117 shutdown_connections 2024-03-16T20:45:11.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.812+0000 7f0b41f14640 1 --2- 172.21.15.122:0/783746117 >> v2:172.21.15.122:3300/0 conn(0x7f0b3c0b90c0 0x7f0b3c0b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.812+0000 7f0b41f14640 1 --2- 172.21.15.122:0/783746117 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b3c144290 0x7f0b3c06ed70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.812+0000 7f0b41f14640 1 -- 172.21.15.122:0/783746117 >> 172.21.15.122:0/783746117 conn(0x7f0b3c070d30 msgr2=0x7f0b3c071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:11.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.812+0000 7f0b41f14640 1 -- 172.21.15.122:0/783746117 shutdown_connections 2024-03-16T20:45:11.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7eff1a21b640 1 Processor -- start 2024-03-16T20:45:11.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7eff1a21b640 1 -- start start 2024-03-16T20:45:11.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7eff1a21b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7eff0c021be0 0x7eff0c0ca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7eff18a18640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7eff0c03ad90 0x7eff0c0214d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58618/0 (socket says 172.21.15.122:58618) 2024-03-16T20:45:11.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7eff18a18640 1 -- 172.21.15.122:0/980935132 learned_addr learned my addr 172.21.15.122:0/980935132 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:11.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7eff1a21b640 1 -- 172.21.15.122:0/980935132 --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0c0caac0 con 0x7eff0c03ad90 2024-03-16T20:45:11.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7eff1a21b640 1 -- 172.21.15.122:0/980935132 --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0c01fa90 con 0x7eff0c00c360 2024-03-16T20:45:11.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7eff1a21b640 1 -- 172.21.15.122:0/980935132 --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff0c0335f0 con 0x7eff0c021be0 2024-03-16T20:45:11.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.812+0000 7f0b41f14640 1 -- 172.21.15.122:0/783746117 wait complete. 2024-03-16T20:45:11.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b41f14640 1 Processor -- start 2024-03-16T20:45:11.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7eff19a1a640 1 --2- 172.21.15.122:0/980935132 >> v2:172.21.15.195:3300/0 conn(0x7eff0c021be0 0x7eff0c0ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7eff0bfff640 1 -- 172.21.15.122:0/980935132 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2982493813 0 0) 0x7eff0c0caac0 con 0x7eff0c03ad90 2024-03-16T20:45:11.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7eff0bfff640 1 -- 172.21.15.122:0/980935132 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff00003530 con 0x7eff0c03ad90 2024-03-16T20:45:11.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7eff0bfff640 1 -- 172.21.15.122:0/980935132 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1009634002 0 0) 0x7eff0c01fa90 con 0x7eff0c00c360 2024-03-16T20:45:11.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7eff0bfff640 1 -- 172.21.15.122:0/980935132 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff0c0caac0 con 0x7eff0c00c360 2024-03-16T20:45:11.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b41f14640 1 -- start start 2024-03-16T20:45:11.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b41f14640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b3c0b90c0 0x7f0b3c1435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b41f14640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b3c144290 0x7f0b3c13c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b41f14640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b3c144c00 0x7f0b3c1ef950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b41f14640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c13beb0 con 0x7f0b3c144290 2024-03-16T20:45:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b41f14640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c13f7f0 con 0x7f0b3c144c00 2024-03-16T20:45:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b41f14640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3c13d880 con 0x7f0b3c0b90c0 2024-03-16T20:45:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b3affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b3c144290 0x7f0b3c13c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b3affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b3c144290 0x7f0b3c13c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41524/0 (socket says 172.21.15.122:41524) 2024-03-16T20:45:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b3affd640 1 -- 172.21.15.122:0/1597041860 learned_addr learned my addr 172.21.15.122:0/1597041860 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b3bfff640 1 --2- 172.21.15.122:0/1597041860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b3c144c00 0x7f0b3c1ef950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b3b7fe640 1 --2- 172.21.15.122:0/1597041860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b3c0b90c0 0x7f0b3c1435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b3affd640 1 -- 172.21.15.122:0/1597041860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b3c0b90c0 msgr2=0x7f0b3c1435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b3affd640 1 --2- 172.21.15.122:0/1597041860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b3c0b90c0 0x7f0b3c1435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b3affd640 1 -- 172.21.15.122:0/1597041860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b3c144c00 msgr2=0x7f0b3c1ef950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b3affd640 1 --2- 172.21.15.122:0/1597041860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b3c144c00 0x7f0b3c1ef950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b3affd640 1 -- 172.21.15.122:0/1597041860 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b300094d0 con 0x7f0b3c144290 2024-03-16T20:45:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b3affd640 1 --2- 172.21.15.122:0/1597041860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b3c144290 0x7f0b3c13c670 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00d7d0 tx=0x7f0b2c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b23fff640 1 -- 172.21.15.122:0/1597041860 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c00b6c0 con 0x7f0b3c144290 2024-03-16T20:45:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b23fff640 1 -- 172.21.15.122:0/1597041860 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c00b820 con 0x7f0b3c144290 2024-03-16T20:45:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b41f14640 1 -- 172.21.15.122:0/1597041860 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b3c13c150 con 0x7f0b3c144290 2024-03-16T20:45:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b23fff640 1 -- 172.21.15.122:0/1597041860 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c00bbd0 con 0x7f0b3c144290 2024-03-16T20:45:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7f0b41f14640 1 -- 172.21.15.122:0/1597041860 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3c138690 con 0x7f0b3c144290 2024-03-16T20:45:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7eff19a1a640 1 -- 172.21.15.122:0/980935132 >> v1:172.21.15.195:6789/0 conn(0x7eff0c00c360 legacy=0x7eff0c03a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7eff0bfff640 1 -- 172.21.15.122:0/980935132 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 388902849 0 0) 0x7eff00003530 con 0x7eff0c03ad90 2024-03-16T20:45:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7eff19a1a640 1 -- 172.21.15.122:0/980935132 >> v1:172.21.15.122:6789/0 conn(0x7eff0c03ad90 legacy=0x7eff0c0214d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.816+0000 7eff19a1a640 1 -- 172.21.15.122:0/980935132 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c00c740 con 0x7eff0c021be0 2024-03-16T20:45:11.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f0b41f14640 1 -- 172.21.15.122:0/1597041860 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b3c0ba6b0 con 0x7f0b3c144290 2024-03-16T20:45:11.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7eff19a1a640 1 --2- 172.21.15.122:0/980935132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c021be0 0x7eff0c0ca580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7eff14000b70 tx=0x7eff14097490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d740db5e69285a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.824 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f0b23fff640 1 -- 172.21.15.122:0/1597041860 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f0b2c00bd30 con 0x7f0b3c144290 2024-03-16T20:45:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7eff0bfff640 1 -- 172.21.15.122:0/980935132 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff10001680 con 0x7eff0c021be0 2024-03-16T20:45:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7eff0bfff640 1 -- 172.21.15.122:0/980935132 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff14122050 con 0x7eff0c021be0 2024-03-16T20:45:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7eff0bfff640 1 -- 172.21.15.122:0/980935132 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff14126400 con 0x7eff0c021be0 2024-03-16T20:45:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f7d8f2bf640 1 Processor -- start 2024-03-16T20:45:11.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f51a606c640 1 Processor -- start 2024-03-16T20:45:11.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f7d8f2bf640 1 -- start start 2024-03-16T20:45:11.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f51a606c640 1 -- start start 2024-03-16T20:45:11.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7eff1a21b640 1 -- 172.21.15.122:0/980935132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c021be0 msgr2=0x7eff0c0ca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7eff1a21b640 1 --2- 172.21.15.122:0/980935132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c021be0 0x7eff0c0ca580 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7eff14000b70 tx=0x7eff14097490 comp rx=0 tx=0).stop 2024-03-16T20:45:11.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f51a606c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f51a0144290 0x7f51a006ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f51a606c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f51a00b90c0 0x7f51a00b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7eff1a21b640 1 -- 172.21.15.122:0/980935132 shutdown_connections 2024-03-16T20:45:11.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7eff1a21b640 1 --2- 172.21.15.122:0/980935132 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c021be0 0x7eff0c0ca580 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7eff1a21b640 1 -- 172.21.15.122:0/980935132 >> 172.21.15.122:0/980935132 conn(0x7eff0c00b5e0 msgr2=0x7eff0c00b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:11.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f51a606c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a006daf0 con 0x7f51a0144c00 2024-03-16T20:45:11.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f51a606c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a00b1390 con 0x7f51a00b90c0 2024-03-16T20:45:11.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f51a606c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a006dd40 con 0x7f51a0144290 2024-03-16T20:45:11.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f0b23fff640 1 --2- 172.21.15.122:0/1597041860 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0b28066e40 0x7f0b28069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f0b3b7fe640 1 --2- 172.21.15.122:0/1597041860 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0b28066e40 0x7f0b28069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f7d8f2bf640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7d80036540 0x7f7d80035560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f7d8f2bf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800c8600 0x7f7d800d3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f7d8f2bf640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d80036e00 con 0x7f7d80006eb0 2024-03-16T20:45:11.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f7d8f2bf640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d80029fb0 con 0x7f7d80036540 2024-03-16T20:45:11.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f7d8f2bf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d800369a0 con 0x7f7d800c8600 2024-03-16T20:45:11.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f519effd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f51a00b90c0 0x7f51a00b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f519ffff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f51a0144c00 0x7f51a013df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58620/0 (socket says 172.21.15.122:58620) 2024-03-16T20:45:11.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f519ffff640 1 -- 172.21.15.122:0/526120455 learned_addr learned my addr 172.21.15.122:0/526120455 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:11.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f519effd640 1 --2- 172.21.15.122:0/526120455 >> v2:172.21.15.122:3300/0 conn(0x7f51a00b90c0 0x7f51a00b7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41532/0 (socket says 172.21.15.122:41532) 2024-03-16T20:45:11.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7eff1a21b640 1 -- 172.21.15.122:0/980935132 shutdown_connections 2024-03-16T20:45:11.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f0b3b7fe640 1 --2- 172.21.15.122:0/1597041860 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0b28066e40 0x7f0b28069300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0b300af000 tx=0x7f0b300b6040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f7d8d85c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800c8600 0x7f7d800d3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f7d8d85c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800c8600 0x7f7d800d3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:34364/0 (socket says 172.21.15.122:34364) 2024-03-16T20:45:11.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f7d8c85a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f7d80036540 0x7f7d80035560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f7d8d85c640 1 -- 172.21.15.122:0/3450354037 learned_addr learned my addr 172.21.15.122:0/3450354037 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:11.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f519effd640 1 -- 172.21.15.122:0/526120455 >> v2:172.21.15.195:3300/0 conn(0x7f51a0144290 msgr2=0x7f51a006ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:11.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7f519f7fe640 1 --2- 172.21.15.122:0/526120455 >> v2:172.21.15.195:3300/0 conn(0x7f51a0144290 0x7f51a006ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f0e64d0f640 1 Processor -- start 2024-03-16T20:45:11.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f519effd640 1 --2- 172.21.15.122:0/526120455 >> v2:172.21.15.195:3300/0 conn(0x7f51a0144290 0x7f51a006ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f519effd640 1 -- 172.21.15.122:0/526120455 >> v1:172.21.15.122:6789/0 conn(0x7f51a0144c00 legacy=0x7f51a013df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f0e64d0f640 1 -- start start 2024-03-16T20:45:11.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f0e64d0f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0e6006ddf0 0x7f0e600b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f0e64d0f640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e600af080 con 0x7f0e6006d290 2024-03-16T20:45:11.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f0e64d0f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e600b81a0 con 0x7f0e6006da10 2024-03-16T20:45:11.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f0e64d0f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e600b8480 con 0x7f0e6006ddf0 2024-03-16T20:45:11.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f0b23fff640 1 -- 172.21.15.122:0/1597041860 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b2c055a30 con 0x7f0b3c144290 2024-03-16T20:45:11.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.820+0000 7eff1a21b640 1 -- 172.21.15.122:0/980935132 wait complete. 2024-03-16T20:45:11.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff1a21b640 1 Processor -- start 2024-03-16T20:45:11.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f519effd640 1 -- 172.21.15.122:0/526120455 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a013e690 con 0x7f51a00b90c0 2024-03-16T20:45:11.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f0e5dd9b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0e6006ddf0 0x7f0e600b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f0e5dd9b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0e6006ddf0 0x7f0e600b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.122:45338/0 (socket says 172.21.15.122:45338) 2024-03-16T20:45:11.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f0e5dd9b640 1 -- 172.21.15.122:0/2860897785 learned_addr learned my addr 172.21.15.122:0/2860897785 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:11.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f7d8c85a640 1 -- 172.21.15.122:0/3450354037 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800c8600 msgr2=0x7f7d800d3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:11.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f7d8c85a640 1 --2- 172.21.15.122:0/3450354037 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800c8600 0x7f7d800d3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f7d8c85a640 1 -- 172.21.15.122:0/3450354037 >> v1:172.21.15.122:6789/0 conn(0x7f7d80006eb0 legacy=0x7f7d80033920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f7d8c85a640 1 -- 172.21.15.122:0/3450354037 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d80035ed0 con 0x7f7d80036540 2024-03-16T20:45:11.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f7d8d85c640 1 --2- 172.21.15.122:0/3450354037 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800c8600 0x7f7d800d3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:45:11.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f519f7fe640 1 --2- 172.21.15.122:0/526120455 >> v2:172.21.15.195:3300/0 conn(0x7f51a0144290 0x7f51a006ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:11.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f0e5dd9b640 1 -- 172.21.15.122:0/2860897785 >> v1:172.21.15.195:6789/0 conn(0x7f0e6006da10 legacy=0x7f0e60143470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-03-16T20:45:11.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f0e5dd9b640 1 -- 172.21.15.122:0/2860897785 >> v1:172.21.15.122:6789/0 conn(0x7f0e6006d290 legacy=0x7f0e600ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:11.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f0e5dd9b640 1 -- 172.21.15.122:0/2860897785 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e600b4a70 con 0x7f0e6006ddf0 2024-03-16T20:45:11.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7fe78ee47640 1 Processor -- start 2024-03-16T20:45:11.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff1a21b640 1 -- start start 2024-03-16T20:45:11.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff1a21b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c00c360 0x7eff0c0228c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff1a21b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0c021be0 0x7eff0c0d50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff1a21b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff0c03ad90 0x7eff0c0d55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff1a21b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c0254e0 con 0x7eff0c03ad90 2024-03-16T20:45:11.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff1a21b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c023d40 con 0x7eff0c00c360 2024-03-16T20:45:11.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff1a21b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c024020 con 0x7eff0c021be0 2024-03-16T20:45:11.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f621ae75640 1 Processor -- start 2024-03-16T20:45:11.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f621ae75640 1 -- start start 2024-03-16T20:45:11.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff19a1a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff0c03ad90 0x7eff0c0d55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff19a1a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff0c03ad90 0x7eff0c0d55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41534/0 (socket says 172.21.15.122:41534) 2024-03-16T20:45:11.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff19a1a640 1 -- 172.21.15.122:0/286061181 learned_addr learned my addr 172.21.15.122:0/286061181 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:11.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff19219640 1 --2- 172.21.15.122:0/286061181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c00c360 0x7eff0c0228c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff18a18640 1 --2- 172.21.15.122:0/286061181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0c021be0 0x7eff0c0d50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f7d8c85a640 1 --2- 172.21.15.122:0/3450354037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d80036540 0x7f7d80035560 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d8802d500 tx=0x7f7d8809b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a35e3cec726d14fb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff19219640 1 -- 172.21.15.122:0/286061181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0c021be0 msgr2=0x7eff0c0d50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff19219640 1 --2- 172.21.15.122:0/286061181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0c021be0 0x7eff0c0d50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff19219640 1 -- 172.21.15.122:0/286061181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff0c03ad90 msgr2=0x7eff0c0d55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff19219640 1 --2- 172.21.15.122:0/286061181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff0c03ad90 0x7eff0c0d55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff19219640 1 -- 172.21.15.122:0/286061181 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff140501f0 con 0x7eff0c00c360 2024-03-16T20:45:11.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff19a1a640 1 --2- 172.21.15.122:0/286061181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff0c03ad90 0x7eff0c0d55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:45:11.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7eff18a18640 1 --2- 172.21.15.122:0/286061181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0c021be0 0x7eff0c0d50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:11.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7fe78ee47640 1 -- start start 2024-03-16T20:45:11.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f621ae75640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f621414c060 0x7f621414f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f7d7ffff640 1 -- 172.21.15.122:0/3450354037 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8812b640 con 0x7f7d80036540 2024-03-16T20:45:11.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f621ae75640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f621414b740 0x7f62140b2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f621ae75640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f621414fb90 con 0x7f621414c440 2024-03-16T20:45:11.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f621ae75640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62140b3220 con 0x7f621414b740 2024-03-16T20:45:11.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.824+0000 7f621ae75640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621414f230 con 0x7f621414c060 2024-03-16T20:45:11.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f6213fff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f621414c440 0x7f62140b7390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58646/0 (socket says 172.21.15.122:58646) 2024-03-16T20:45:11.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f6213fff640 1 -- 172.21.15.122:0/4085369193 learned_addr learned my addr 172.21.15.122:0/4085369193 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:11.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7eff19219640 1 --2- 172.21.15.122:0/286061181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c00c360 0x7eff0c0228c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7eff0400c3c0 tx=0x7eff0400c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f6218c11640 1 --2- 172.21.15.122:0/4085369193 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f621414c060 0x7f621414f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f6219412640 1 --2- 172.21.15.122:0/4085369193 >> v2:172.21.15.195:3300/0 conn(0x7f621414b740 0x7f62140b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7eff09ffb640 1 -- 172.21.15.122:0/286061181 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff040052d0 con 0x7eff0c00c360 2024-03-16T20:45:11.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7eff09ffb640 1 -- 172.21.15.122:0/286061181 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff04005450 con 0x7eff0c00c360 2024-03-16T20:45:11.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7fe78ee47640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe788066fa0 0x7fe7880b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7fe78ee47640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe78814faf0 con 0x7fe7880af260 2024-03-16T20:45:11.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7fe78ee47640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7880b4990 con 0x7fe7880b4560 2024-03-16T20:45:11.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7fe78ee47640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7880aeab0 con 0x7fe788066fa0 2024-03-16T20:45:11.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d7ffff640 1 -- 172.21.15.122:0/3450354037 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8812b7a0 con 0x7f7d80036540 2024-03-16T20:45:11.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d7ffff640 1 -- 172.21.15.122:0/3450354037 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8812bbd0 con 0x7f7d80036540 2024-03-16T20:45:11.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d8f2bf640 1 -- 172.21.15.122:0/3450354037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d80036540 msgr2=0x7f7d80035560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d8f2bf640 1 --2- 172.21.15.122:0/3450354037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d80036540 0x7f7d80035560 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7d8802d500 tx=0x7f7d8809b860 comp rx=0 tx=0).stop 2024-03-16T20:45:11.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f6218c11640 1 -- 172.21.15.122:0/4085369193 >> v2:172.21.15.195:3300/0 conn(0x7f621414b740 msgr2=0x7f62140b2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:11.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f6218c11640 1 --2- 172.21.15.122:0/4085369193 >> v2:172.21.15.195:3300/0 conn(0x7f621414b740 0x7f62140b2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f6218c11640 1 -- 172.21.15.122:0/4085369193 >> v1:172.21.15.122:6789/0 conn(0x7f621414c440 legacy=0x7f62140b7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f6218c11640 1 -- 172.21.15.122:0/4085369193 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621414ee50 con 0x7f621414c060 2024-03-16T20:45:11.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d8f2bf640 1 -- 172.21.15.122:0/3450354037 shutdown_connections 2024-03-16T20:45:11.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d8f2bf640 1 --2- 172.21.15.122:0/3450354037 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d800c8600 0x7f7d800d3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d8f2bf640 1 --2- 172.21.15.122:0/3450354037 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d80036540 0x7f7d80035560 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d8f2bf640 1 -- 172.21.15.122:0/3450354037 >> 172.21.15.122:0/3450354037 conn(0x7f7d8003a860 msgr2=0x7f7d8003ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:11.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7fe78cbe3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe788066fa0 0x7fe7880b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7fe78cbe3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe788066fa0 0x7fe7880b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.122:45364/0 (socket says 172.21.15.122:45364) 2024-03-16T20:45:11.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7fe78cbe3640 1 -- 172.21.15.122:0/2372550347 learned_addr learned my addr 172.21.15.122:0/2372550347 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:11.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7fe78d3e4640 1 --1- 172.21.15.122:0/2372550347 >> v1:172.21.15.195:6789/0 conn(0x7fe7880b4560 0x7fe78814f3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.122:51302/0 (socket says 172.21.15.122:51302) 2024-03-16T20:45:11.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d8f2bf640 1 -- 172.21.15.122:0/3450354037 shutdown_connections 2024-03-16T20:45:11.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f519effd640 1 --2- 172.21.15.122:0/526120455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a00b90c0 0x7f51a00b7550 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f51a0067d00 tx=0x7f519001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4794d1cbd89ee4a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f0e5dd9b640 1 --2- 172.21.15.122:0/2860897785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e6006ddf0 0x7f0e600b3110 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0e50006fd0 tx=0x7f0e5001e870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f42fce020c405f30 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f0e5d59a640 1 -- 172.21.15.122:0/2860897785 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e500a8680 con 0x7f0e6006ddf0 2024-03-16T20:45:11.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7eff09ffb640 1 -- 172.21.15.122:0/286061181 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff04003840 con 0x7eff0c00c360 2024-03-16T20:45:11.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7eff1a21b640 1 -- 172.21.15.122:0/286061181 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0c024300 con 0x7eff0c00c360 2024-03-16T20:45:11.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7eff1a21b640 1 -- 172.21.15.122:0/286061181 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0c022e00 con 0x7eff0c00c360 2024-03-16T20:45:11.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d8f2bf640 1 -- 172.21.15.122:0/3450354037 wait complete. 2024-03-16T20:45:11.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d8f2bf640 1 Processor -- start 2024-03-16T20:45:11.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7eff1a21b640 1 -- 172.21.15.122:0/286061181 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0c024520 con 0x7eff0c00c360 2024-03-16T20:45:11.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f0e5d59a640 1 -- 172.21.15.122:0/2860897785 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e500a87e0 con 0x7f0e6006ddf0 2024-03-16T20:45:11.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f0e5d59a640 1 -- 172.21.15.122:0/2860897785 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e500a8b70 con 0x7f0e6006ddf0 2024-03-16T20:45:11.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f0e64d0f640 1 -- 172.21.15.122:0/2860897785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e6006ddf0 msgr2=0x7f0e600b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f0e64d0f640 1 --2- 172.21.15.122:0/2860897785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e6006ddf0 0x7f0e600b3110 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0e50006fd0 tx=0x7f0e5001e870 comp rx=0 tx=0).stop 2024-03-16T20:45:11.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f519e7fc640 1 -- 172.21.15.122:0/526120455 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51900ad680 con 0x7f51a00b90c0 2024-03-16T20:45:11.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f0e64d0f640 1 -- 172.21.15.122:0/2860897785 shutdown_connections 2024-03-16T20:45:11.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f0e64d0f640 1 --2- 172.21.15.122:0/2860897785 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e6006ddf0 0x7f0e600b3110 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f0e64d0f640 1 -- 172.21.15.122:0/2860897785 >> 172.21.15.122:0/2860897785 conn(0x7f0e6009a4c0 msgr2=0x7f0e6009a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:11.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f0e64d0f640 1 -- 172.21.15.122:0/2860897785 shutdown_connections 2024-03-16T20:45:11.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f0b23fff640 1 -- 172.21.15.122:0/1597041860 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0b2c051cd0 con 0x7f0b3c144290 2024-03-16T20:45:11.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f519e7fc640 1 -- 172.21.15.122:0/526120455 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51900ad7e0 con 0x7f51a00b90c0 2024-03-16T20:45:11.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d8f2bf640 1 -- start start 2024-03-16T20:45:11.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d8f2bf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d80006eb0 0x7f7d80032e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d8f2bf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d80036540 0x7f7d800ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d8f2bf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d800c8600 0x7f7d800d3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.848 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d8f2bf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d800cc6c0 con 0x7f7d800c8600 2024-03-16T20:45:11.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d8f2bf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d800c9ed0 con 0x7f7d80006eb0 2024-03-16T20:45:11.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f7d8f2bf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d800cbaf0 con 0x7f7d80036540 2024-03-16T20:45:11.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f7d8d85c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d800c8600 0x7f7d800d3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f7d8d85c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d800c8600 0x7f7d800d3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41544/0 (socket says 172.21.15.122:41544) 2024-03-16T20:45:11.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f7d8d85c640 1 -- 172.21.15.122:0/1556138480 learned_addr learned my addr 172.21.15.122:0/1556138480 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:11.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f7d8d05b640 1 --2- 172.21.15.122:0/1556138480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d80006eb0 0x7f7d80032e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f7d8c85a640 1 --2- 172.21.15.122:0/1556138480 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d80036540 0x7f7d800ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f51a606c640 1 -- 172.21.15.122:0/526120455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a00b90c0 msgr2=0x7f51a00b7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f51a606c640 1 --2- 172.21.15.122:0/526120455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a00b90c0 0x7f51a00b7550 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f51a0067d00 tx=0x7f519001c3a0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f6218c11640 1 --2- 172.21.15.122:0/4085369193 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f621414c060 0x7f621414f650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f62000097c0 tx=0x7f620001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83e9dddfce25d3f5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f7d8d85c640 1 -- 172.21.15.122:0/1556138480 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d80036540 msgr2=0x7f7d800ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f7d8d85c640 1 --2- 172.21.15.122:0/1556138480 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d80036540 0x7f7d800ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f7d8d85c640 1 -- 172.21.15.122:0/1556138480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d80006eb0 msgr2=0x7f7d80032e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f7d8d85c640 1 --2- 172.21.15.122:0/1556138480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d80006eb0 0x7f7d80032e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f7d8d85c640 1 -- 172.21.15.122:0/1556138480 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8400b580 con 0x7f7d800c8600 2024-03-16T20:45:11.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f7d8d85c640 1 --2- 172.21.15.122:0/1556138480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d800c8600 0x7f7d800d3600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400fae0 tx=0x7f7d8400d350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7fe78cbe3640 1 -- 172.21.15.122:0/2372550347 >> v1:172.21.15.195:6789/0 conn(0x7fe7880b4560 legacy=0x7fe78814f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7f0e64d0f640 1 -- 172.21.15.122:0/2860897785 wait complete. 2024-03-16T20:45:11.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e64d0f640 1 Processor -- start 2024-03-16T20:45:11.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7fe78cbe3640 1 -- 172.21.15.122:0/2372550347 >> v1:172.21.15.122:6789/0 conn(0x7fe7880af260 legacy=0x7fe78814ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.828+0000 7fe78cbe3640 1 -- 172.21.15.122:0/2372550347 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7880afa90 con 0x7fe788066fa0 2024-03-16T20:45:11.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f7d7dffb640 1 -- 172.21.15.122:0/1556138480 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d84004020 con 0x7f7d800c8600 2024-03-16T20:45:11.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f7d7dffb640 1 -- 172.21.15.122:0/1556138480 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d84004180 con 0x7f7d800c8600 2024-03-16T20:45:11.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f62137fe640 1 -- 172.21.15.122:0/4085369193 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62000a4ea0 con 0x7f621414c060 2024-03-16T20:45:11.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f62137fe640 1 -- 172.21.15.122:0/4085369193 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62000a7050 con 0x7f621414c060 2024-03-16T20:45:11.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f62137fe640 1 -- 172.21.15.122:0/4085369193 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62000ab820 con 0x7f621414c060 2024-03-16T20:45:11.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f621ae75640 1 -- 172.21.15.122:0/4085369193 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f621414c060 msgr2=0x7f621414f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f621ae75640 1 --2- 172.21.15.122:0/4085369193 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f621414c060 0x7f621414f650 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f62000097c0 tx=0x7f620001c410 comp rx=0 tx=0).stop 2024-03-16T20:45:11.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e64d0f640 1 -- start start 2024-03-16T20:45:11.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f621ae75640 1 -- 172.21.15.122:0/4085369193 shutdown_connections 2024-03-16T20:45:11.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f621ae75640 1 --2- 172.21.15.122:0/4085369193 >> v2:172.21.15.195:3300/0 conn(0x7f621414b740 0x7f62140b2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f621ae75640 1 --2- 172.21.15.122:0/4085369193 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f621414c060 0x7f621414f650 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f62000097c0 tx=0x7f620001c410 comp rx=0 tx=0).stop 2024-03-16T20:45:11.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f621ae75640 1 -- 172.21.15.122:0/4085369193 >> 172.21.15.122:0/4085369193 conn(0x7f62140b9490 msgr2=0x7f62140b98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:11.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f7d7dffb640 1 -- 172.21.15.122:0/1556138480 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d84004510 con 0x7f7d800c8600 2024-03-16T20:45:11.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f7d8f2bf640 1 -- 172.21.15.122:0/1556138480 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d880501f0 con 0x7f7d800c8600 2024-03-16T20:45:11.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f7d8f2bf640 1 -- 172.21.15.122:0/1556138480 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d800cab80 con 0x7f7d800c8600 2024-03-16T20:45:11.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7eff09ffb640 1 -- 172.21.15.122:0/286061181 <== mon.1 v2:172.21.15.195:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7eff040039a0 con 0x7eff0c00c360 2024-03-16T20:45:11.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f621ae75640 1 -- 172.21.15.122:0/4085369193 shutdown_connections 2024-03-16T20:45:11.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f621ae75640 1 -- 172.21.15.122:0/4085369193 wait complete. 2024-03-16T20:45:11.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e64d0f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e6006d290 0x7f0e600b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e64d0f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e6006da10 0x7f0e601ec280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e64d0f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e6006ddf0 0x7f0e600aff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e64d0f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e601ec7c0 con 0x7f0e6006da10 2024-03-16T20:45:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e64d0f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e600b1cf0 con 0x7f0e6006d290 2024-03-16T20:45:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e64d0f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e601eca30 con 0x7f0e6006ddf0 2024-03-16T20:45:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e5ed9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e6006ddf0 0x7f0e600aff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e5ed9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e6006ddf0 0x7f0e600aff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:34406/0 (socket says 172.21.15.122:34406) 2024-03-16T20:45:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e5ed9d640 1 -- 172.21.15.122:0/2294264085 learned_addr learned my addr 172.21.15.122:0/2294264085 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e5dd9b640 1 --2- 172.21.15.122:0/2294264085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e6006da10 0x7f0e601ec280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7eff09ffb640 1 --2- 172.21.15.122:0/286061181 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efef8066e40 0x7efef8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7eff09ffb640 1 -- 172.21.15.122:0/286061181 <== mon.1 v2:172.21.15.195:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff04084340 con 0x7eff0c00c360 2024-03-16T20:45:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e5dd9b640 1 -- 172.21.15.122:0/2294264085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e6006ddf0 msgr2=0x7f0e600aff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e5dd9b640 1 --2- 172.21.15.122:0/2294264085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e6006ddf0 0x7f0e600aff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e5dd9b640 1 -- 172.21.15.122:0/2294264085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e6006d290 msgr2=0x7f0e600b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e5dd9b640 1 --2- 172.21.15.122:0/2294264085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e6006d290 0x7f0e600b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e5dd9b640 1 -- 172.21.15.122:0/2294264085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e50008500 con 0x7f0e6006da10 2024-03-16T20:45:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f0e5dd9b640 1 --2- 172.21.15.122:0/2294264085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e6006da10 0x7f0e601ec280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0e500ae000 tx=0x7f0e5001b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f0e4effd640 1 -- 172.21.15.122:0/2294264085 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e50005ba0 con 0x7f0e6006da10 2024-03-16T20:45:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7fe78cbe3640 1 --2- 172.21.15.122:0/2372550347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe788066fa0 0x7fe7880b4020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe780009e50 tx=0x7fe78001c930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1abce484638a38e1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f0e64d0f640 1 -- 172.21.15.122:0/2294264085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e601ecd10 con 0x7f0e6006da10 2024-03-16T20:45:11.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f0e64d0f640 1 -- 172.21.15.122:0/2294264085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e600b1a50 con 0x7f0e6006da10 2024-03-16T20:45:11.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f0e4effd640 1 -- 172.21.15.122:0/2294264085 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e50005d00 con 0x7f0e6006da10 2024-03-16T20:45:11.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f0e4effd640 1 -- 172.21.15.122:0/2294264085 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e500a8b90 con 0x7f0e6006da10 2024-03-16T20:45:11.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f7d7dffb640 1 -- 172.21.15.122:0/1556138480 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f7d84003620 con 0x7f7d800c8600 2024-03-16T20:45:11.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f621ae75640 1 Processor -- start 2024-03-16T20:45:11.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f7d7dffb640 1 --2- 172.21.15.122:0/1556138480 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7d6c066e40 0x7f7d6c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f621ae75640 1 -- start start 2024-03-16T20:45:11.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f7d7dffb640 1 -- 172.21.15.122:0/1556138480 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7d84085190 con 0x7f7d800c8600 2024-03-16T20:45:11.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f7d8d05b640 1 --2- 172.21.15.122:0/1556138480 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7d6c066e40 0x7f7d6c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f0e64d0f640 1 -- 172.21.15.122:0/2294264085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e440052a0 con 0x7f0e6006da10 2024-03-16T20:45:11.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f0e4effd640 1 -- 172.21.15.122:0/2294264085 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f0e500a8cf0 con 0x7f0e6006da10 2024-03-16T20:45:11.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f0e4effd640 1 --2- 172.21.15.122:0/2294264085 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0e40066e70 0x7f0e40069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f7d8d05b640 1 --2- 172.21.15.122:0/1556138480 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7d6c066e40 0x7f7d6c069300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d78001810 tx=0x7f7d78009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f0e4effd640 1 -- 172.21.15.122:0/2294264085 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e5011ed60 con 0x7f0e6006da10 2024-03-16T20:45:11.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f0e5e59c640 1 --2- 172.21.15.122:0/2294264085 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0e40066e70 0x7f0e40069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f0e5e59c640 1 --2- 172.21.15.122:0/2294264085 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0e40066e70 0x7f0e40069330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e540096f0 tx=0x7f0e54006c50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f621ae75640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f621414b740 0x7f621414dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f621ae75640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f621414c440 0x7f62140a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f621ae75640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62140a1630 0x7f62140b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f621ae75640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62140a1a80 con 0x7f621414c440 2024-03-16T20:45:11.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f621ae75640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621409fad0 con 0x7f62140a1630 2024-03-16T20:45:11.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f621ae75640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62140a07f0 con 0x7f621414b740 2024-03-16T20:45:11.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f6213fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f621414c440 0x7f62140a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f6218c11640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f621414b740 0x7f621414dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f6213fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f621414c440 0x7f62140a10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41582/0 (socket says 172.21.15.122:41582) 2024-03-16T20:45:11.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f6213fff640 1 -- 172.21.15.122:0/3505389601 learned_addr learned my addr 172.21.15.122:0/3505389601 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:11.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f6213fff640 1 -- 172.21.15.122:0/3505389601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f621414b740 msgr2=0x7f621414dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f6213fff640 1 --2- 172.21.15.122:0/3505389601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f621414b740 0x7f621414dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f6213fff640 1 -- 172.21.15.122:0/3505389601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62140a1630 msgr2=0x7f62140b52c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T20:45:11.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f6213fff640 1 --2- 172.21.15.122:0/3505389601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62140a1630 0x7f62140b52c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f6213fff640 1 -- 172.21.15.122:0/3505389601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6200009420 con 0x7f621414c440 2024-03-16T20:45:11.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f6213fff640 1 --2- 172.21.15.122:0/3505389601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f621414c440 0x7f62140a10f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f620801d410 tx=0x7f620801d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.840+0000 7eff18a18640 1 --2- 172.21.15.122:0/286061181 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efef8066e40 0x7efef8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f51a606c640 1 -- 172.21.15.122:0/526120455 shutdown_connections 2024-03-16T20:45:11.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f51a606c640 1 --2- 172.21.15.122:0/526120455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a00b90c0 0x7f51a00b7550 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f51a606c640 1 --2- 172.21.15.122:0/526120455 >> v2:172.21.15.195:3300/0 conn(0x7f51a0144290 0x7f51a006ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f51a606c640 1 -- 172.21.15.122:0/526120455 >> 172.21.15.122:0/526120455 conn(0x7f51a0070d30 msgr2=0x7f51a0071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:11.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f51a606c640 1 -- 172.21.15.122:0/526120455 shutdown_connections 2024-03-16T20:45:11.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f51a606c640 1 -- 172.21.15.122:0/526120455 wait complete. 2024-03-16T20:45:11.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f51a606c640 1 Processor -- start 2024-03-16T20:45:11.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.832+0000 7f51a606c640 1 -- start start 2024-03-16T20:45:11.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f51a606c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a00b90c0 0x7f51a0150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f51a606c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51a0144290 0x7f51a01e8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f51a606c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51a0144c00 0x7f51a0139710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f51a606c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a0151330 con 0x7f51a00b90c0 2024-03-16T20:45:11.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f51a606c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a0138fa0 con 0x7f51a0144290 2024-03-16T20:45:11.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f51a606c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a013a840 con 0x7f51a0144c00 2024-03-16T20:45:11.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f519f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a00b90c0 0x7f51a0150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f519f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a00b90c0 0x7f51a0150b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41566/0 (socket says 172.21.15.122:41566) 2024-03-16T20:45:11.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f519f7fe640 1 -- 172.21.15.122:0/2846831017 learned_addr learned my addr 172.21.15.122:0/2846831017 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:11.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.840+0000 7f7d8f2bf640 1 -- 172.21.15.122:0/1556138480 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d800ca360 con 0x7f7d800c8600 2024-03-16T20:45:11.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f519ffff640 1 --2- 172.21.15.122:0/2846831017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51a0144c00 0x7f51a0139710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f519f7fe640 1 -- 172.21.15.122:0/2846831017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51a0144c00 msgr2=0x7f51a0139710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f519f7fe640 1 --2- 172.21.15.122:0/2846831017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51a0144c00 0x7f51a0139710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f519f7fe640 1 -- 172.21.15.122:0/2846831017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51a0144290 msgr2=0x7f51a01e8ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-16T20:45:11.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f519f7fe640 1 --2- 172.21.15.122:0/2846831017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51a0144290 0x7f51a01e8ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f519f7fe640 1 -- 172.21.15.122:0/2846831017 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5190009420 con 0x7f51a00b90c0 2024-03-16T20:45:11.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.836+0000 7f519f7fe640 1 --2- 172.21.15.122:0/2846831017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a00b90c0 0x7f51a0150b80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f519400c7f0 tx=0x7f519400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.840+0000 7eff18a18640 1 --2- 172.21.15.122:0/286061181 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efef8066e40 0x7efef8069300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0365f0 tx=0x7eff10009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.840+0000 7f5187fff640 1 -- 172.21.15.122:0/2846831017 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194007820 con 0x7f51a00b90c0 2024-03-16T20:45:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.840+0000 7f5187fff640 1 -- 172.21.15.122:0/2846831017 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5194007980 con 0x7f51a00b90c0 2024-03-16T20:45:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.840+0000 7f51a606c640 1 -- 172.21.15.122:0/2846831017 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a01391f0 con 0x7f51a00b90c0 2024-03-16T20:45:11.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.840+0000 7f0e4effd640 1 -- 172.21.15.122:0/2294264085 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0e500ebb40 con 0x7f0e6006da10 2024-03-16T20:45:11.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.840+0000 7f62117fa640 1 -- 172.21.15.122:0/3505389601 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6208005af0 con 0x7f621414c440 2024-03-16T20:45:11.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.840+0000 7f621ae75640 1 -- 172.21.15.122:0/3505389601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621409f7f0 con 0x7f621414c440 2024-03-16T20:45:11.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.840+0000 7f621ae75640 1 -- 172.21.15.122:0/3505389601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62140a0ad0 con 0x7f621414c440 2024-03-16T20:45:11.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.844+0000 7f621ae75640 1 -- 172.21.15.122:0/3505389601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f80052a0 con 0x7f621414c440 2024-03-16T20:45:11.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.844+0000 7fe7877fe640 1 -- 172.21.15.122:0/2372550347 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7800a55f0 con 0x7fe788066fa0 2024-03-16T20:45:11.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.844+0000 7fe7877fe640 1 -- 172.21.15.122:0/2372550347 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7800a5750 con 0x7fe788066fa0 2024-03-16T20:45:11.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.844+0000 7fe7877fe640 1 -- 172.21.15.122:0/2372550347 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7800a5b20 con 0x7fe788066fa0 2024-03-16T20:45:11.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.844+0000 7eff09ffb640 1 -- 172.21.15.122:0/286061181 <== mon.1 v2:172.21.15.195:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7eff04051140 con 0x7eff0c00c360 2024-03-16T20:45:11.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.844+0000 7f5187fff640 1 -- 172.21.15.122:0/2846831017 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5194007d30 con 0x7f51a00b90c0 2024-03-16T20:45:11.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7fe78ee47640 1 -- 172.21.15.122:0/2372550347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe788066fa0 msgr2=0x7fe7880b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7fe78ee47640 1 --2- 172.21.15.122:0/2372550347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe788066fa0 0x7fe7880b4020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe780009e50 tx=0x7fe78001c930 comp rx=0 tx=0).stop 2024-03-16T20:45:11.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7f51a606c640 1 -- 172.21.15.122:0/2846831017 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a01452f0 con 0x7f51a00b90c0 2024-03-16T20:45:11.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7fe78ee47640 1 -- 172.21.15.122:0/2372550347 shutdown_connections 2024-03-16T20:45:11.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7fe78ee47640 1 --2- 172.21.15.122:0/2372550347 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe788066fa0 0x7fe7880b4020 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7fe78ee47640 1 -- 172.21.15.122:0/2372550347 >> 172.21.15.122:0/2372550347 conn(0x7fe78806d2b0 msgr2=0x7fe78806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:11.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7fe78ee47640 1 -- 172.21.15.122:0/2372550347 shutdown_connections 2024-03-16T20:45:11.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7f51a606c640 1 -- 172.21.15.122:0/2846831017 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a00ba6b0 con 0x7f51a00b90c0 2024-03-16T20:45:11.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7f7d7dffb640 1 -- 172.21.15.122:0/1556138480 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7d800ca360 con 0x7f7d800c8600 2024-03-16T20:45:11.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7fe78ee47640 1 -- 172.21.15.122:0/2372550347 wait complete. 2024-03-16T20:45:11.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7f62117fa640 1 -- 172.21.15.122:0/3505389601 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6208005c50 con 0x7f621414c440 2024-03-16T20:45:11.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7f62117fa640 1 -- 172.21.15.122:0/3505389601 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6208007270 con 0x7f621414c440 2024-03-16T20:45:11.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7fe78ee47640 1 Processor -- start 2024-03-16T20:45:11.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7fe78ee47640 1 -- start start 2024-03-16T20:45:11.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7fe78ee47640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe788066fa0 0x7fe7880b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7fe78ee47640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7880af260 0x7fe7881232c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7fe78ee47640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7880b4560 0x7fe788123800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7fe78ee47640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7880b1e30 con 0x7fe7880af260 2024-03-16T20:45:11.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7fe78ee47640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe788124970 con 0x7fe7880b4560 2024-03-16T20:45:11.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7fe78ee47640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe788122b50 con 0x7fe788066fa0 2024-03-16T20:45:11.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7f5187fff640 1 -- 172.21.15.122:0/2846831017 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f5194024480 con 0x7f51a00b90c0 2024-03-16T20:45:11.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7f5187fff640 1 --2- 172.21.15.122:0/2846831017 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f518c066e40 0x7f518c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7f5187fff640 1 -- 172.21.15.122:0/2846831017 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5194084ff0 con 0x7f51a00b90c0 2024-03-16T20:45:11.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.848+0000 7fe787fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7880af260 0x7fe7881232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7fe787fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7880af260 0x7fe7881232c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41594/0 (socket says 172.21.15.122:41594) 2024-03-16T20:45:11.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7fe787fff640 1 -- 172.21.15.122:0/74717511 learned_addr learned my addr 172.21.15.122:0/74717511 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:11.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7fe78d3e4640 1 --2- 172.21.15.122:0/74717511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7880b4560 0x7fe788123800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7fe78cbe3640 1 --2- 172.21.15.122:0/74717511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe788066fa0 0x7fe7880b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7f62117fa640 1 -- 172.21.15.122:0/3505389601 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f62080073d0 con 0x7f621414c440 2024-03-16T20:45:11.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7f62117fa640 1 --2- 172.21.15.122:0/3505389601 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6204066f80 0x7f6204069440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7fe78cbe3640 1 -- 172.21.15.122:0/74717511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7880b4560 msgr2=0x7fe788123800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7fe78cbe3640 1 --2- 172.21.15.122:0/74717511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7880b4560 0x7fe788123800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7fe78cbe3640 1 -- 172.21.15.122:0/74717511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7880af260 msgr2=0x7fe7881232c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:11.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7fe78cbe3640 1 --2- 172.21.15.122:0/74717511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7880af260 0x7fe7881232c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:11.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7fe78cbe3640 1 -- 172.21.15.122:0/74717511 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe780009b30 con 0x7fe788066fa0 2024-03-16T20:45:11.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7f6218c11640 1 --2- 172.21.15.122:0/3505389601 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6204066f80 0x7f6204069440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7fe78cbe3640 1 --2- 172.21.15.122:0/74717511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe788066fa0 0x7fe7880b0880 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe7800ab000 tx=0x7fe7800196d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7fe7857fa640 1 -- 172.21.15.122:0/74717511 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7800b8470 con 0x7fe788066fa0 2024-03-16T20:45:11.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7f6218c11640 1 --2- 172.21.15.122:0/3505389601 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6204066f80 0x7f6204069440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f62000ad000 tx=0x7f6200008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7fe7857fa640 1 -- 172.21.15.122:0/74717511 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7800b85d0 con 0x7fe788066fa0 2024-03-16T20:45:11.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7fe78ee47640 1 -- 172.21.15.122:0/74717511 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe788122e30 con 0x7fe788066fa0 2024-03-16T20:45:11.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.852+0000 7fe78ee47640 1 -- 172.21.15.122:0/74717511 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe788122920 con 0x7fe788066fa0 2024-03-16T20:45:11.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.856+0000 7fe7857fa640 1 -- 172.21.15.122:0/74717511 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7800b8980 con 0x7fe788066fa0 2024-03-16T20:45:11.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.856+0000 7fe78ee47640 1 -- 172.21.15.122:0/74717511 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe788074bb0 con 0x7fe788066fa0 2024-03-16T20:45:11.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.856+0000 7fe7857fa640 1 -- 172.21.15.122:0/74717511 <== mon.2 v2:172.21.15.195:3301/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe7800b8ba0 con 0x7fe788066fa0 2024-03-16T20:45:11.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.856+0000 7fe7857fa640 1 --2- 172.21.15.122:0/74717511 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe774066e40 0x7fe774069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:11.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.856+0000 7fe7857fa640 1 -- 172.21.15.122:0/74717511 <== mon.2 v2:172.21.15.195:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe78011f960 con 0x7fe788066fa0 2024-03-16T20:45:11.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.856+0000 7fe787fff640 1 --2- 172.21.15.122:0/74717511 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe774066e40 0x7fe774069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.856+0000 7fe787fff640 1 --2- 172.21.15.122:0/74717511 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe774066e40 0x7fe774069300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe7880b2980 tx=0x7fe77c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.856+0000 7f62117fa640 1 -- 172.21.15.122:0/3505389601 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6208005e40 con 0x7f621414c440 2024-03-16T20:45:11.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.856+0000 7f5187fff640 1 -- 172.21.15.122:0/2846831017 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5194052000 con 0x7f51a00b90c0 2024-03-16T20:45:11.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.860+0000 7f519effd640 1 --2- 172.21.15.122:0/2846831017 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f518c066e40 0x7f518c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:11.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.860+0000 7f519effd640 1 --2- 172.21.15.122:0/2846831017 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f518c066e40 0x7f518c069300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f51900af000 tx=0x7f51900a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:11.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.868+0000 7f62117fa640 1 -- 172.21.15.122:0/3505389601 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6208062330 con 0x7f621414c440 2024-03-16T20:45:11.879 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:11.868+0000 7fe7857fa640 1 -- 172.21.15.122:0/74717511 <== mon.2 v2:172.21.15.195:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe7800ec890 con 0x7fe788066fa0 2024-03-16T20:45:12.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.124+0000 7f9da3bfe640 1 -- 172.21.15.122:0/1447573112 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9d84005820 con 0x7f9d9c0b5570 2024-03-16T20:45:12.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.124+0000 7f9d927fc640 1 -- 172.21.15.122:0/1447573112 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9d8c014090 con 0x7f9d9c0b5570 2024-03-16T20:45:12.131 INFO:teuthology.orchestra.run.smithi122.stdout:55834574854 2024-03-16T20:45:12.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.128+0000 7f9d7ffff640 1 -- 172.21.15.122:0/1447573112 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9d80066ec0 msgr2=0x7f9d80069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.128+0000 7f9d7ffff640 1 --2- 172.21.15.122:0/1447573112 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9d80066ec0 0x7f9d80069380 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d940ae000 tx=0x7f9d940a8000 comp rx=0 tx=0).stop 2024-03-16T20:45:12.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.128+0000 7f9d7ffff640 1 -- 172.21.15.122:0/1447573112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d9c0b5570 msgr2=0x7f9d9c0716c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.128+0000 7f9d7ffff640 1 --2- 172.21.15.122:0/1447573112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d9c0b5570 0x7f9d9c0716c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c00b550 tx=0x7f9d8c00ba10 comp rx=0 tx=0).stop 2024-03-16T20:45:12.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.128+0000 7f9d7ffff640 1 -- 172.21.15.122:0/1447573112 shutdown_connections 2024-03-16T20:45:12.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.128+0000 7f9d7ffff640 1 --2- 172.21.15.122:0/1447573112 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9d80066ec0 0x7f9d80069380 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.128+0000 7f9d7ffff640 1 --2- 172.21.15.122:0/1447573112 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d9c140930 0x7f9d9c1522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.128+0000 7f9d7ffff640 1 --2- 172.21.15.122:0/1447573112 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9d9c0b5570 0x7f9d9c0716c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.128+0000 7f9d7ffff640 1 --2- 172.21.15.122:0/1447573112 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9d9c0b5190 0x7f9d9c137760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.128+0000 7f9d7ffff640 1 -- 172.21.15.122:0/1447573112 >> 172.21.15.122:0/1447573112 conn(0x7f9d9c09de70 msgr2=0x7f9d9c074320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:12.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.128+0000 7f9d7ffff640 1 -- 172.21.15.122:0/1447573112 shutdown_connections 2024-03-16T20:45:12.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.128+0000 7f9d7ffff640 1 -- 172.21.15.122:0/1447573112 wait complete. 2024-03-16T20:45:12.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.140+0000 7f0e64d0f640 1 -- 172.21.15.122:0/2294264085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0e44005550 con 0x7f0e6006da10 2024-03-16T20:45:12.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.140+0000 7f0e4effd640 1 -- 172.21.15.122:0/2294264085 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0e500ad090 con 0x7f0e6006da10 2024-03-16T20:45:12.145 INFO:teuthology.orchestra.run.smithi122.stdout:55834574855 2024-03-16T20:45:12.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.144+0000 7f0e4cff9640 1 -- 172.21.15.122:0/2294264085 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0e40066e70 msgr2=0x7f0e40069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.144+0000 7f0e4cff9640 1 --2- 172.21.15.122:0/2294264085 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0e40066e70 0x7f0e40069330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0e540096f0 tx=0x7f0e54006c50 comp rx=0 tx=0).stop 2024-03-16T20:45:12.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.144+0000 7f0e4cff9640 1 -- 172.21.15.122:0/2294264085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e6006da10 msgr2=0x7f0e601ec280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.144+0000 7f0e4cff9640 1 --2- 172.21.15.122:0/2294264085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e6006da10 0x7f0e601ec280 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0e500ae000 tx=0x7f0e5001b610 comp rx=0 tx=0).stop 2024-03-16T20:45:12.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.144+0000 7f0e4cff9640 1 -- 172.21.15.122:0/2294264085 shutdown_connections 2024-03-16T20:45:12.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.144+0000 7f0e4cff9640 1 --2- 172.21.15.122:0/2294264085 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0e40066e70 0x7f0e40069330 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.144+0000 7f0e4cff9640 1 --2- 172.21.15.122:0/2294264085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0e6006ddf0 0x7f0e600aff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.144+0000 7f0e4cff9640 1 --2- 172.21.15.122:0/2294264085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e6006da10 0x7f0e601ec280 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.144+0000 7f0e4cff9640 1 --2- 172.21.15.122:0/2294264085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0e6006d290 0x7f0e600b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.144+0000 7f0e4cff9640 1 -- 172.21.15.122:0/2294264085 >> 172.21.15.122:0/2294264085 conn(0x7f0e6009a4c0 msgr2=0x7f0e600b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:12.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.144+0000 7f0e4cff9640 1 -- 172.21.15.122:0/2294264085 shutdown_connections 2024-03-16T20:45:12.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.144+0000 7f0e4cff9640 1 -- 172.21.15.122:0/2294264085 wait complete. 2024-03-16T20:45:12.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.144+0000 7f0b41f14640 1 -- 172.21.15.122:0/1597041860 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0b3c074bb0 con 0x7f0b3c144290 2024-03-16T20:45:12.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.148+0000 7f0b23fff640 1 -- 172.21.15.122:0/1597041860 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0b2c055390 con 0x7f0b3c144290 2024-03-16T20:45:12.150 INFO:teuthology.orchestra.run.smithi122.stdout:55834574854 2024-03-16T20:45:12.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.152+0000 7f0b21ffb640 1 -- 172.21.15.122:0/1597041860 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0b28066e40 msgr2=0x7f0b28069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.152+0000 7f0b21ffb640 1 --2- 172.21.15.122:0/1597041860 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0b28066e40 0x7f0b28069300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0b300af000 tx=0x7f0b300b6040 comp rx=0 tx=0).stop 2024-03-16T20:45:12.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.152+0000 7f0b21ffb640 1 -- 172.21.15.122:0/1597041860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b3c144290 msgr2=0x7f0b3c13c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.152+0000 7f0b21ffb640 1 --2- 172.21.15.122:0/1597041860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b3c144290 0x7f0b3c13c670 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c00d7d0 tx=0x7f0b2c00dc90 comp rx=0 tx=0).stop 2024-03-16T20:45:12.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-03-16T20:45:12.155 DEBUG:teuthology.parallel:result is None 2024-03-16T20:45:12.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.152+0000 7f0b21ffb640 1 -- 172.21.15.122:0/1597041860 shutdown_connections 2024-03-16T20:45:12.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.152+0000 7f0b21ffb640 1 --2- 172.21.15.122:0/1597041860 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0b28066e40 0x7f0b28069300 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.152+0000 7f0b21ffb640 1 --2- 172.21.15.122:0/1597041860 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0b3c144c00 0x7f0b3c1ef950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.152+0000 7f0b21ffb640 1 --2- 172.21.15.122:0/1597041860 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b3c144290 0x7f0b3c13c670 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.152+0000 7f0b21ffb640 1 --2- 172.21.15.122:0/1597041860 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0b3c0b90c0 0x7f0b3c1435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.152+0000 7f0b21ffb640 1 -- 172.21.15.122:0/1597041860 >> 172.21.15.122:0/1597041860 conn(0x7f0b3c070d30 msgr2=0x7f0b3c13dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:12.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.152+0000 7f0b21ffb640 1 -- 172.21.15.122:0/1597041860 shutdown_connections 2024-03-16T20:45:12.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.152+0000 7f0b21ffb640 1 -- 172.21.15.122:0/1597041860 wait complete. 2024-03-16T20:45:12.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.160+0000 7eff1a21b640 1 -- 172.21.15.122:0/286061181 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7eff0c0057f0 con 0x7eff0c00c360 2024-03-16T20:45:12.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.160+0000 7f7d8f2bf640 1 -- 172.21.15.122:0/1556138480 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7d800ca360 con 0x7f7d800c8600 2024-03-16T20:45:12.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.160+0000 7eff09ffb640 1 -- 172.21.15.122:0/286061181 <== mon.1 v2:172.21.15.195:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7eff04059e60 con 0x7eff0c00c360 2024-03-16T20:45:12.164 INFO:teuthology.orchestra.run.smithi122.stdout:55834574854 2024-03-16T20:45:12.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.160+0000 7f7d7dffb640 1 -- 172.21.15.122:0/1556138480 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7d800ca360 con 0x7f7d800c8600 2024-03-16T20:45:12.164 INFO:teuthology.orchestra.run.smithi122.stdout:55834574854 2024-03-16T20:45:12.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7f7d8f2bf640 1 -- 172.21.15.122:0/1556138480 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7d6c066e40 msgr2=0x7f7d6c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7f7d8f2bf640 1 --2- 172.21.15.122:0/1556138480 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7d6c066e40 0x7f7d6c069300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7d78001810 tx=0x7f7d78009040 comp rx=0 tx=0).stop 2024-03-16T20:45:12.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7f7d8f2bf640 1 -- 172.21.15.122:0/1556138480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d800c8600 msgr2=0x7f7d800d3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7f7d8f2bf640 1 --2- 172.21.15.122:0/1556138480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d800c8600 0x7f7d800d3600 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400fae0 tx=0x7f7d8400d350 comp rx=0 tx=0).stop 2024-03-16T20:45:12.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7f7d8f2bf640 1 -- 172.21.15.122:0/1556138480 shutdown_connections 2024-03-16T20:45:12.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7f7d8f2bf640 1 --2- 172.21.15.122:0/1556138480 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7d6c066e40 0x7f7d6c069300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7f7d8f2bf640 1 --2- 172.21.15.122:0/1556138480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7d800c8600 0x7f7d800d3600 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7f7d8f2bf640 1 --2- 172.21.15.122:0/1556138480 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7d80036540 0x7f7d800ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7f7d8f2bf640 1 --2- 172.21.15.122:0/1556138480 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7d80006eb0 0x7f7d80032e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7f7d8f2bf640 1 -- 172.21.15.122:0/1556138480 >> 172.21.15.122:0/1556138480 conn(0x7f7d8003a860 msgr2=0x7f7d80006650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:12.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7f7d8f2bf640 1 -- 172.21.15.122:0/1556138480 shutdown_connections 2024-03-16T20:45:12.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7f7d8f2bf640 1 -- 172.21.15.122:0/1556138480 wait complete. 2024-03-16T20:45:12.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7eff1a21b640 1 -- 172.21.15.122:0/286061181 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efef8066e40 msgr2=0x7efef8069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7eff1a21b640 1 --2- 172.21.15.122:0/286061181 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efef8066e40 0x7efef8069300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0365f0 tx=0x7eff10009040 comp rx=0 tx=0).stop 2024-03-16T20:45:12.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7eff1a21b640 1 -- 172.21.15.122:0/286061181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c00c360 msgr2=0x7eff0c0228c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7eff1a21b640 1 --2- 172.21.15.122:0/286061181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c00c360 0x7eff0c0228c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7eff0400c3c0 tx=0x7eff0400c880 comp rx=0 tx=0).stop 2024-03-16T20:45:12.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7eff1a21b640 1 -- 172.21.15.122:0/286061181 shutdown_connections 2024-03-16T20:45:12.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7eff1a21b640 1 --2- 172.21.15.122:0/286061181 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efef8066e40 0x7efef8069300 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7eff1a21b640 1 --2- 172.21.15.122:0/286061181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff0c03ad90 0x7eff0c0d55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7eff1a21b640 1 --2- 172.21.15.122:0/286061181 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0c021be0 0x7eff0c0d50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7eff1a21b640 1 --2- 172.21.15.122:0/286061181 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0c00c360 0x7eff0c0228c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7eff1a21b640 1 -- 172.21.15.122:0/286061181 >> 172.21.15.122:0/286061181 conn(0x7eff0c00b5e0 msgr2=0x7eff0c005a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:12.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7eff1a21b640 1 -- 172.21.15.122:0/286061181 shutdown_connections 2024-03-16T20:45:12.169 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.164+0000 7eff1a21b640 1 -- 172.21.15.122:0/286061181 wait complete. 2024-03-16T20:45:12.175 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2024-03-16T20:45:12.175 DEBUG:teuthology.parallel:result is None 2024-03-16T20:45:12.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.172+0000 7f51a606c640 1 -- 172.21.15.122:0/2846831017 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f51a006f080 con 0x7f51a00b90c0 2024-03-16T20:45:12.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.176+0000 7f5187fff640 1 -- 172.21.15.122:0/2846831017 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f519401f090 con 0x7f51a00b90c0 2024-03-16T20:45:12.177 INFO:teuthology.orchestra.run.smithi122.stdout:55834574854 2024-03-16T20:45:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.176+0000 7f51a606c640 1 -- 172.21.15.122:0/2846831017 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f518c066e40 msgr2=0x7f518c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.176+0000 7f51a606c640 1 --2- 172.21.15.122:0/2846831017 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f518c066e40 0x7f518c069300 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f51900af000 tx=0x7f51900a7000 comp rx=0 tx=0).stop 2024-03-16T20:45:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.176+0000 7f51a606c640 1 -- 172.21.15.122:0/2846831017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a00b90c0 msgr2=0x7f51a0150b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.176+0000 7f51a606c640 1 --2- 172.21.15.122:0/2846831017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a00b90c0 0x7f51a0150b80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f519400c7f0 tx=0x7f519400ccb0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.176+0000 7f51a606c640 1 -- 172.21.15.122:0/2846831017 shutdown_connections 2024-03-16T20:45:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.176+0000 7f51a606c640 1 --2- 172.21.15.122:0/2846831017 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f518c066e40 0x7f518c069300 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.176+0000 7f51a606c640 1 --2- 172.21.15.122:0/2846831017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f51a0144c00 0x7f51a0139710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.176+0000 7f51a606c640 1 --2- 172.21.15.122:0/2846831017 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f51a0144290 0x7f51a01e8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.176+0000 7f51a606c640 1 --2- 172.21.15.122:0/2846831017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f51a00b90c0 0x7f51a0150b80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.176+0000 7f51a606c640 1 -- 172.21.15.122:0/2846831017 >> 172.21.15.122:0/2846831017 conn(0x7f51a0070d30 msgr2=0x7f51a0071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.176+0000 7f51a606c640 1 -- 172.21.15.122:0/2846831017 shutdown_connections 2024-03-16T20:45:12.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.176+0000 7f51a606c640 1 -- 172.21.15.122:0/2846831017 wait complete. 2024-03-16T20:45:12.183 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-03-16T20:45:12.183 DEBUG:teuthology.parallel:result is None 2024-03-16T20:45:12.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.184+0000 7f621ae75640 1 -- 172.21.15.122:0/3505389601 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f61f8005820 con 0x7f621414c440 2024-03-16T20:45:12.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.184+0000 7f62117fa640 1 -- 172.21.15.122:0/3505389601 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6208032020 con 0x7f621414c440 2024-03-16T20:45:12.188 INFO:teuthology.orchestra.run.smithi122.stdout:55834574855 2024-03-16T20:45:12.190 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2024-03-16T20:45:12.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.184+0000 7fe78ee47640 1 -- 172.21.15.122:0/74717511 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe7880aef10 con 0x7fe788066fa0 2024-03-16T20:45:12.190 DEBUG:teuthology.parallel:result is None 2024-03-16T20:45:12.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7fe7857fa640 1 -- 172.21.15.122:0/74717511 <== mon.2 v2:172.21.15.195:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe7800bc020 con 0x7fe788066fa0 2024-03-16T20:45:12.191 INFO:teuthology.orchestra.run.smithi122.stdout:55834574854 2024-03-16T20:45:12.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7f621ae75640 1 -- 172.21.15.122:0/3505389601 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6204066f80 msgr2=0x7f6204069440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7f621ae75640 1 --2- 172.21.15.122:0/3505389601 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6204066f80 0x7f6204069440 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f62000ad000 tx=0x7f6200008040 comp rx=0 tx=0).stop 2024-03-16T20:45:12.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7f621ae75640 1 -- 172.21.15.122:0/3505389601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f621414c440 msgr2=0x7f62140a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7f621ae75640 1 --2- 172.21.15.122:0/3505389601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f621414c440 0x7f62140a10f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f620801d410 tx=0x7f620801d8d0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7f621ae75640 1 -- 172.21.15.122:0/3505389601 shutdown_connections 2024-03-16T20:45:12.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7f621ae75640 1 --2- 172.21.15.122:0/3505389601 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6204066f80 0x7f6204069440 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7f621ae75640 1 --2- 172.21.15.122:0/3505389601 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62140a1630 0x7f62140b52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7f621ae75640 1 --2- 172.21.15.122:0/3505389601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f621414c440 0x7f62140a10f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7f621ae75640 1 --2- 172.21.15.122:0/3505389601 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f621414b740 0x7f621414dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7f621ae75640 1 -- 172.21.15.122:0/3505389601 >> 172.21.15.122:0/3505389601 conn(0x7f62140b9490 msgr2=0x7f62140ba7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:12.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7f621ae75640 1 -- 172.21.15.122:0/3505389601 shutdown_connections 2024-03-16T20:45:12.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7f621ae75640 1 -- 172.21.15.122:0/3505389601 wait complete. 2024-03-16T20:45:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7fe78ee47640 1 -- 172.21.15.122:0/74717511 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe774066e40 msgr2=0x7fe774069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7fe78ee47640 1 --2- 172.21.15.122:0/74717511 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe774066e40 0x7fe774069300 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe7880b2980 tx=0x7fe77c009040 comp rx=0 tx=0).stop 2024-03-16T20:45:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7fe78ee47640 1 -- 172.21.15.122:0/74717511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe788066fa0 msgr2=0x7fe7880b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7fe78ee47640 1 --2- 172.21.15.122:0/74717511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe788066fa0 0x7fe7880b0880 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe7800ab000 tx=0x7fe7800196d0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7fe78ee47640 1 -- 172.21.15.122:0/74717511 shutdown_connections 2024-03-16T20:45:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7fe78ee47640 1 --2- 172.21.15.122:0/74717511 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe774066e40 0x7fe774069300 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7fe78ee47640 1 --2- 172.21.15.122:0/74717511 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7880b4560 0x7fe788123800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7fe78ee47640 1 --2- 172.21.15.122:0/74717511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7880af260 0x7fe7881232c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7fe78ee47640 1 --2- 172.21.15.122:0/74717511 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe788066fa0 0x7fe7880b0880 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7fe78ee47640 1 -- 172.21.15.122:0/74717511 >> 172.21.15.122:0/74717511 conn(0x7fe78806d2b0 msgr2=0x7fe788143a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7fe78ee47640 1 -- 172.21.15.122:0/74717511 shutdown_connections 2024-03-16T20:45:12.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.188+0000 7fe78ee47640 1 -- 172.21.15.122:0/74717511 wait complete. 2024-03-16T20:45:12.195 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2024-03-16T20:45:12.195 DEBUG:teuthology.parallel:result is None 2024-03-16T20:45:12.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2024-03-16T20:45:12.201 DEBUG:teuthology.parallel:result is None 2024-03-16T20:45:12.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2024-03-16T20:45:12.216 DEBUG:teuthology.parallel:result is None 2024-03-16T20:45:12.216 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-03-16T20:45:12.217 DEBUG:teuthology.parallel:result is None 2024-03-16T20:45:12.217 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-16T20:45:12.217 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T20:45:12.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.356+0000 7f882eef7640 1 Processor -- start 2024-03-16T20:45:12.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.356+0000 7f882eef7640 1 -- start start 2024-03-16T20:45:12.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882eef7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8828148500 0x7f882813ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:12.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882eef7640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f882813f350 0x7f882814f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:12.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882eef7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88280716b0 con 0x7f882806edd0 2024-03-16T20:45:12.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882eef7640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8828072570 con 0x7f882813f350 2024-03-16T20:45:12.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882eef7640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f882813f800 con 0x7f8828148500 2024-03-16T20:45:12.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882d494640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f882813f350 0x7f882814f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:12.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882d494640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f882813f350 0x7f882814f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41602/0 (socket says 172.21.15.122:41602) 2024-03-16T20:45:12.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882d494640 1 -- 172.21.15.122:0/881089829 learned_addr learned my addr 172.21.15.122:0/881089829 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:12.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f881ffff640 1 --2- 172.21.15.122:0/881089829 >> v2:172.21.15.195:3300/0 conn(0x7f8828148500 0x7f882813ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:12.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882d494640 1 -- 172.21.15.122:0/881089829 >> v2:172.21.15.195:3300/0 conn(0x7f8828148500 msgr2=0x7f882813ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:12.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882d494640 1 --2- 172.21.15.122:0/881089829 >> v2:172.21.15.195:3300/0 conn(0x7f8828148500 0x7f882813ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882d494640 1 -- 172.21.15.122:0/881089829 >> v1:172.21.15.195:6789/0 conn(0x7f882806edd0 legacy=0x7f8828071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:12.363 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882d494640 1 -- 172.21.15.122:0/881089829 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882806f1b0 con 0x7f882813f350 2024-03-16T20:45:12.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882d494640 1 --2- 172.21.15.122:0/881089829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f882813f350 0x7f882814f880 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f882806bfd0 tx=0x7f882001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e1d8bb133202ad server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:12.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f881f7fe640 1 -- 172.21.15.122:0/881089829 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88200a5890 con 0x7f882813f350 2024-03-16T20:45:12.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f881f7fe640 1 -- 172.21.15.122:0/881089829 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88200a59f0 con 0x7f882813f350 2024-03-16T20:45:12.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f881f7fe640 1 -- 172.21.15.122:0/881089829 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88200a5d80 con 0x7f882813f350 2024-03-16T20:45:12.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882eef7640 1 -- 172.21.15.122:0/881089829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f882813f350 msgr2=0x7f882814f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882eef7640 1 --2- 172.21.15.122:0/881089829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f882813f350 0x7f882814f880 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f882806bfd0 tx=0x7f882001c9b0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882eef7640 1 -- 172.21.15.122:0/881089829 shutdown_connections 2024-03-16T20:45:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882eef7640 1 --2- 172.21.15.122:0/881089829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f882813f350 0x7f882814f880 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882eef7640 1 --2- 172.21.15.122:0/881089829 >> v2:172.21.15.195:3300/0 conn(0x7f8828148500 0x7f882813ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.360+0000 7f882eef7640 1 -- 172.21.15.122:0/881089829 >> 172.21.15.122:0/881089829 conn(0x7f882806c0a0 msgr2=0x7f882806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882eef7640 1 -- 172.21.15.122:0/881089829 shutdown_connections 2024-03-16T20:45:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882eef7640 1 -- 172.21.15.122:0/881089829 wait complete. 2024-03-16T20:45:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882eef7640 1 Processor -- start 2024-03-16T20:45:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882eef7640 1 -- start start 2024-03-16T20:45:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882eef7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f882806edd0 0x7f8828147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882eef7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f882813f350 0x7f882813a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882eef7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8828148500 0x7f88281385a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882eef7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882813fba0 con 0x7f882806edd0 2024-03-16T20:45:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882eef7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88281e0aa0 con 0x7f8828148500 2024-03-16T20:45:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882eef7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88281e0d80 con 0x7f882813f350 2024-03-16T20:45:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882cc93640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f882806edd0 0x7f8828147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882cc93640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f882806edd0 0x7f8828147730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41608/0 (socket says 172.21.15.122:41608) 2024-03-16T20:45:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882cc93640 1 -- 172.21.15.122:0/1316624563 learned_addr learned my addr 172.21.15.122:0/1316624563 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882d494640 1 --2- 172.21.15.122:0/1316624563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8828148500 0x7f88281385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f881ffff640 1 --2- 172.21.15.122:0/1316624563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f882813f350 0x7f882813a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882cc93640 1 -- 172.21.15.122:0/1316624563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f882813f350 msgr2=0x7f882813a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882cc93640 1 --2- 172.21.15.122:0/1316624563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f882813f350 0x7f882813a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882cc93640 1 -- 172.21.15.122:0/1316624563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8828148500 msgr2=0x7f88281385a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882cc93640 1 --2- 172.21.15.122:0/1316624563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8828148500 0x7f88281385a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882cc93640 1 -- 172.21.15.122:0/1316624563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8820009b30 con 0x7f882806edd0 2024-03-16T20:45:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882cc93640 1 --2- 172.21.15.122:0/1316624563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f882806edd0 0x7f8828147730 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f882400c450 tx=0x7f882400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f881d7fa640 1 -- 172.21.15.122:0/1316624563 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8824012830 con 0x7f882806edd0 2024-03-16T20:45:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f881d7fa640 1 -- 172.21.15.122:0/1316624563 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88240129b0 con 0x7f882806edd0 2024-03-16T20:45:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f881d7fa640 1 -- 172.21.15.122:0/1316624563 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8824012d60 con 0x7f882806edd0 2024-03-16T20:45:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882eef7640 1 -- 172.21.15.122:0/1316624563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88281e1060 con 0x7f882806edd0 2024-03-16T20:45:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882eef7640 1 -- 172.21.15.122:0/1316624563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8828147e00 con 0x7f882806edd0 2024-03-16T20:45:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f882eef7640 1 -- 172.21.15.122:0/1316624563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88281e0910 con 0x7f882806edd0 2024-03-16T20:45:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.364+0000 7f881d7fa640 1 -- 172.21.15.122:0/1316624563 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f882401c460 con 0x7f882806edd0 2024-03-16T20:45:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.368+0000 7f881d7fa640 1 --2- 172.21.15.122:0/1316624563 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8814066e40 0x7f8814069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.368+0000 7f881d7fa640 1 -- 172.21.15.122:0/1316624563 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88240847e0 con 0x7f882806edd0 2024-03-16T20:45:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.368+0000 7f881ffff640 1 --2- 172.21.15.122:0/1316624563 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8814066e40 0x7f8814069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.368+0000 7f881ffff640 1 --2- 172.21.15.122:0/1316624563 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8814066e40 0x7f8814069300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8828130c70 tx=0x7f8818008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:12.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.372+0000 7f881d7fa640 1 -- 172.21.15.122:0/1316624563 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8824051660 con 0x7f882806edd0 2024-03-16T20:45:12.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.516+0000 7f882eef7640 1 -- 172.21.15.122:0/1316624563 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f88280b3a70 con 0x7f8814066e40 2024-03-16T20:45:12.537 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.532+0000 7f881d7fa640 1 -- 172.21.15.122:0/1316624563 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251908 (secure 0 0 0) 0x7f881800be80 con 0x7f8814066e40 2024-03-16T20:45:12.540 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:12.541 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-03-16T20:45:12.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.540+0000 7f882eef7640 1 -- 172.21.15.122:0/1316624563 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8814066e40 msgr2=0x7f8814069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.540+0000 7f882eef7640 1 --2- 172.21.15.122:0/1316624563 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8814066e40 0x7f8814069300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8828130c70 tx=0x7f8818008040 comp rx=0 tx=0).stop 2024-03-16T20:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.540+0000 7f882eef7640 1 -- 172.21.15.122:0/1316624563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f882806edd0 msgr2=0x7f8828147730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.540+0000 7f882eef7640 1 --2- 172.21.15.122:0/1316624563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f882806edd0 0x7f8828147730 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f882400c450 tx=0x7f882400c910 comp rx=0 tx=0).stop 2024-03-16T20:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.540+0000 7f882eef7640 1 -- 172.21.15.122:0/1316624563 shutdown_connections 2024-03-16T20:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.540+0000 7f882eef7640 1 --2- 172.21.15.122:0/1316624563 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8814066e40 0x7f8814069300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.540+0000 7f882eef7640 1 --2- 172.21.15.122:0/1316624563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8828148500 0x7f88281385a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.540+0000 7f882eef7640 1 --2- 172.21.15.122:0/1316624563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f882813f350 0x7f882813a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.540+0000 7f882eef7640 1 --2- 172.21.15.122:0/1316624563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f882806edd0 0x7f8828147730 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.540+0000 7f882eef7640 1 -- 172.21.15.122:0/1316624563 >> 172.21.15.122:0/1316624563 conn(0x7f882806c0a0 msgr2=0x7f88280b4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.540+0000 7f882eef7640 1 -- 172.21.15.122:0/1316624563 shutdown_connections 2024-03-16T20:45:12.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.540+0000 7f882eef7640 1 -- 172.21.15.122:0/1316624563 wait complete. 2024-03-16T20:45:12.557 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-16T20:45:10.578599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235752,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738968,"statfs":{"total":773094113280,"available":772852703232,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.233375"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459997+0000","last_change":"2024-03-16T20:45:05.428065+0000","last_active":"2024-03-16T20:45:06.459997+0000","last_peered":"2024-03-16T20:45:06.459997+0000","last_clean":"2024-03-16T20:45:06.459997+0000","last_became_active":"2024-03-16T20:45:05.427498+0000","last_became_peered":"2024-03-16T20:45:05.427498+0000","last_unstale":"2024-03-16T20:45:06.459997+0000","last_undegraded":"2024-03-16T20:45:06.459997+0000","last_fullsized":"2024-03-16T20:45:06.459997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:40:17.773165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410546+0000","last_change":"2024-03-16T20:45:05.418257+0000","last_active":"2024-03-16T20:45:06.410546+0000","last_peered":"2024-03-16T20:45:06.410546+0000","last_clean":"2024-03-16T20:45:06.410546+0000","last_became_active":"2024-03-16T20:45:05.416660+0000","last_became_peered":"2024-03-16T20:45:05.416660+0000","last_unstale":"2024-03-16T20:45:06.410546+0000","last_undegraded":"2024-03-16T20:45:06.410546+0000","last_fullsized":"2024-03-16T20:45:06.410546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:29:10.023600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337420+0000","last_change":"2024-03-16T20:45:05.410148+0000","last_active":"2024-03-16T20:45:07.337420+0000","last_peered":"2024-03-16T20:45:07.337420+0000","last_clean":"2024-03-16T20:45:07.337420+0000","last_became_active":"2024-03-16T20:45:05.409853+0000","last_became_peered":"2024-03-16T20:45:05.409853+0000","last_unstale":"2024-03-16T20:45:07.337420+0000","last_undegraded":"2024-03-16T20:45:07.337420+0000","last_fullsized":"2024-03-16T20:45:07.337420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:23:41.936524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412946+0000","last_change":"2024-03-16T20:45:05.419996+0000","last_active":"2024-03-16T20:45:06.412946+0000","last_peered":"2024-03-16T20:45:06.412946+0000","last_clean":"2024-03-16T20:45:06.412946+0000","last_became_active":"2024-03-16T20:45:05.417478+0000","last_became_peered":"2024-03-16T20:45:05.417478+0000","last_unstale":"2024-03-16T20:45:06.412946+0000","last_undegraded":"2024-03-16T20:45:06.412946+0000","last_fullsized":"2024-03-16T20:45:06.412946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:26:54.432519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459267+0000","last_change":"2024-03-16T20:45:05.422236+0000","last_active":"2024-03-16T20:45:06.459267+0000","last_peered":"2024-03-16T20:45:06.459267+0000","last_clean":"2024-03-16T20:45:06.459267+0000","last_became_active":"2024-03-16T20:45:05.421698+0000","last_became_peered":"2024-03-16T20:45:05.421698+0000","last_unstale":"2024-03-16T20:45:06.459267+0000","last_undegraded":"2024-03-16T20:45:06.459267+0000","last_fullsized":"2024-03-16T20:45:06.459267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:54.931097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412907+0000","last_change":"2024-03-16T20:45:05.419865+0000","last_active":"2024-03-16T20:45:06.412907+0000","last_peered":"2024-03-16T20:45:06.412907+0000","last_clean":"2024-03-16T20:45:06.412907+0000","last_became_active":"2024-03-16T20:45:05.419476+0000","last_became_peered":"2024-03-16T20:45:05.419476+0000","last_unstale":"2024-03-16T20:45:06.412907+0000","last_undegraded":"2024-03-16T20:45:06.412907+0000","last_fullsized":"2024-03-16T20:45:06.412907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:50:22.920748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.458816+0000","last_change":"2024-03-16T20:45:05.421986+0000","last_active":"2024-03-16T20:45:06.458816+0000","last_peered":"2024-03-16T20:45:06.458816+0000","last_clean":"2024-03-16T20:45:06.458816+0000","last_became_active":"2024-03-16T20:45:05.421125+0000","last_became_peered":"2024-03-16T20:45:05.421125+0000","last_unstale":"2024-03-16T20:45:06.458816+0000","last_undegraded":"2024-03-16T20:45:06.458816+0000","last_fullsized":"2024-03-16T20:45:06.458816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:26:54.432519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.999419+0000","last_change":"2024-03-16T20:45:05.402329+0000","last_active":"2024-03-16T20:45:06.999419+0000","last_peered":"2024-03-16T20:45:06.999419+0000","last_clean":"2024-03-16T20:45:06.999419+0000","last_became_active":"2024-03-16T20:45:05.400677+0000","last_became_peered":"2024-03-16T20:45:05.400677+0000","last_unstale":"2024-03-16T20:45:06.999419+0000","last_undegraded":"2024-03-16T20:45:06.999419+0000","last_fullsized":"2024-03-16T20:45:06.999419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:22:42.961272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.460167+0000","last_change":"2024-03-16T20:45:05.425819+0000","last_active":"2024-03-16T20:45:06.460167+0000","last_peered":"2024-03-16T20:45:06.460167+0000","last_clean":"2024-03-16T20:45:06.460167+0000","last_became_active":"2024-03-16T20:45:05.425238+0000","last_became_peered":"2024-03-16T20:45:05.425238+0000","last_unstale":"2024-03-16T20:45:06.460167+0000","last_undegraded":"2024-03-16T20:45:06.460167+0000","last_fullsized":"2024-03-16T20:45:06.460167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:04:42.305935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412787+0000","last_change":"2024-03-16T20:45:05.419949+0000","last_active":"2024-03-16T20:45:06.412787+0000","last_peered":"2024-03-16T20:45:06.412787+0000","last_clean":"2024-03-16T20:45:06.412787+0000","last_became_active":"2024-03-16T20:45:05.419670+0000","last_became_peered":"2024-03-16T20:45:05.419670+0000","last_unstale":"2024-03-16T20:45:06.412787+0000","last_undegraded":"2024-03-16T20:45:06.412787+0000","last_fullsized":"2024-03-16T20:45:06.412787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:35:07.902704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412340+0000","last_change":"2024-03-16T20:45:05.424035+0000","last_active":"2024-03-16T20:45:06.412340+0000","last_peered":"2024-03-16T20:45:06.412340+0000","last_clean":"2024-03-16T20:45:06.412340+0000","last_became_active":"2024-03-16T20:45:05.423657+0000","last_became_peered":"2024-03-16T20:45:05.423657+0000","last_unstale":"2024-03-16T20:45:06.412340+0000","last_undegraded":"2024-03-16T20:45:06.412340+0000","last_fullsized":"2024-03-16T20:45:06.412340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:03:04.229071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.336271+0000","last_change":"2024-03-16T20:45:05.427007+0000","last_active":"2024-03-16T20:45:07.336271+0000","last_peered":"2024-03-16T20:45:07.336271+0000","last_clean":"2024-03-16T20:45:07.336271+0000","last_became_active":"2024-03-16T20:45:05.426482+0000","last_became_peered":"2024-03-16T20:45:05.426482+0000","last_unstale":"2024-03-16T20:45:07.336271+0000","last_undegraded":"2024-03-16T20:45:07.336271+0000","last_fullsized":"2024-03-16T20:45:07.336271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:20:09.822409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.999992+0000","last_change":"2024-03-16T20:45:05.418866+0000","last_active":"2024-03-16T20:45:06.999992+0000","last_peered":"2024-03-16T20:45:06.999992+0000","last_clean":"2024-03-16T20:45:06.999992+0000","last_became_active":"2024-03-16T20:45:05.418458+0000","last_became_peered":"2024-03-16T20:45:05.418458+0000","last_unstale":"2024-03-16T20:45:06.999992+0000","last_undegraded":"2024-03-16T20:45:06.999992+0000","last_fullsized":"2024-03-16T20:45:06.999992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:35:07.902704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337384+0000","last_change":"2024-03-16T20:45:05.411615+0000","last_active":"2024-03-16T20:45:07.337384+0000","last_peered":"2024-03-16T20:45:07.337384+0000","last_clean":"2024-03-16T20:45:07.337384+0000","last_became_active":"2024-03-16T20:45:05.410789+0000","last_became_peered":"2024-03-16T20:45:05.410789+0000","last_unstale":"2024-03-16T20:45:07.337384+0000","last_undegraded":"2024-03-16T20:45:07.337384+0000","last_fullsized":"2024-03-16T20:45:07.337384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:08:02.898870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410139+0000","last_change":"2024-03-16T20:45:05.411051+0000","last_active":"2024-03-16T20:45:06.410139+0000","last_peered":"2024-03-16T20:45:06.410139+0000","last_clean":"2024-03-16T20:45:06.410139+0000","last_became_active":"2024-03-16T20:45:05.409805+0000","last_became_peered":"2024-03-16T20:45:05.409805+0000","last_unstale":"2024-03-16T20:45:06.410139+0000","last_undegraded":"2024-03-16T20:45:06.410139+0000","last_fullsized":"2024-03-16T20:45:06.410139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:26:51.023909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.411744+0000","last_change":"2024-03-16T20:45:05.413320+0000","last_active":"2024-03-16T20:45:06.411744+0000","last_peered":"2024-03-16T20:45:06.411744+0000","last_clean":"2024-03-16T20:45:06.411744+0000","last_became_active":"2024-03-16T20:45:05.412473+0000","last_became_peered":"2024-03-16T20:45:05.412473+0000","last_unstale":"2024-03-16T20:45:06.411744+0000","last_undegraded":"2024-03-16T20:45:06.411744+0000","last_fullsized":"2024-03-16T20:45:06.411744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:35:07.902704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.414480+0000","last_change":"2024-03-16T20:45:05.421924+0000","last_active":"2024-03-16T20:45:06.414480+0000","last_peered":"2024-03-16T20:45:06.414480+0000","last_clean":"2024-03-16T20:45:06.414480+0000","last_became_active":"2024-03-16T20:45:05.421632+0000","last_became_peered":"2024-03-16T20:45:05.421632+0000","last_unstale":"2024-03-16T20:45:06.414480+0000","last_undegraded":"2024-03-16T20:45:06.414480+0000","last_fullsized":"2024-03-16T20:45:06.414480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:18:48.237612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337803+0000","last_change":"2024-03-16T20:45:05.424802+0000","last_active":"2024-03-16T20:45:07.337803+0000","last_peered":"2024-03-16T20:45:07.337803+0000","last_clean":"2024-03-16T20:45:07.337803+0000","last_became_active":"2024-03-16T20:45:05.424388+0000","last_became_peered":"2024-03-16T20:45:05.424388+0000","last_unstale":"2024-03-16T20:45:07.337803+0000","last_undegraded":"2024-03-16T20:45:07.337803+0000","last_fullsized":"2024-03-16T20:45:07.337803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:20:09.822409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.460115+0000","last_change":"2024-03-16T20:45:05.418605+0000","last_active":"2024-03-16T20:45:06.460115+0000","last_peered":"2024-03-16T20:45:06.460115+0000","last_clean":"2024-03-16T20:45:06.460115+0000","last_became_active":"2024-03-16T20:45:05.418293+0000","last_became_peered":"2024-03-16T20:45:05.418293+0000","last_unstale":"2024-03-16T20:45:06.460115+0000","last_undegraded":"2024-03-16T20:45:06.460115+0000","last_fullsized":"2024-03-16T20:45:06.460115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:57:45.943288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412374+0000","last_change":"2024-03-16T20:45:05.422968+0000","last_active":"2024-03-16T20:45:06.412374+0000","last_peered":"2024-03-16T20:45:06.412374+0000","last_clean":"2024-03-16T20:45:06.412374+0000","last_became_active":"2024-03-16T20:45:05.422612+0000","last_became_peered":"2024-03-16T20:45:05.422612+0000","last_unstale":"2024-03-16T20:45:06.412374+0000","last_undegraded":"2024-03-16T20:45:06.412374+0000","last_fullsized":"2024-03-16T20:45:06.412374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:26:51.023909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337190+0000","last_change":"2024-03-16T20:45:05.426370+0000","last_active":"2024-03-16T20:45:07.337190+0000","last_peered":"2024-03-16T20:45:07.337190+0000","last_clean":"2024-03-16T20:45:07.337190+0000","last_became_active":"2024-03-16T20:45:05.426035+0000","last_became_peered":"2024-03-16T20:45:05.426035+0000","last_unstale":"2024-03-16T20:45:07.337190+0000","last_undegraded":"2024-03-16T20:45:07.337190+0000","last_fullsized":"2024-03-16T20:45:07.337190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:06:46.922690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413393+0000","last_change":"2024-03-16T20:45:05.410178+0000","last_active":"2024-03-16T20:45:06.413393+0000","last_peered":"2024-03-16T20:45:06.413393+0000","last_clean":"2024-03-16T20:45:06.413393+0000","last_became_active":"2024-03-16T20:45:05.409867+0000","last_became_peered":"2024-03-16T20:45:05.409867+0000","last_unstale":"2024-03-16T20:45:06.413393+0000","last_undegraded":"2024-03-16T20:45:06.413393+0000","last_fullsized":"2024-03-16T20:45:06.413393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:54.931097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.999686+0000","last_change":"2024-03-16T20:45:05.413326+0000","last_active":"2024-03-16T20:45:06.999686+0000","last_peered":"2024-03-16T20:45:06.999686+0000","last_clean":"2024-03-16T20:45:06.999686+0000","last_became_active":"2024-03-16T20:45:05.412843+0000","last_became_peered":"2024-03-16T20:45:05.412843+0000","last_unstale":"2024-03-16T20:45:06.999686+0000","last_undegraded":"2024-03-16T20:45:06.999686+0000","last_fullsized":"2024-03-16T20:45:06.999686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:03:04.229071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412143+0000","last_change":"2024-03-16T20:45:05.404852+0000","last_active":"2024-03-16T20:45:06.412143+0000","last_peered":"2024-03-16T20:45:06.412143+0000","last_clean":"2024-03-16T20:45:06.412143+0000","last_became_active":"2024-03-16T20:45:05.403728+0000","last_became_peered":"2024-03-16T20:45:05.403728+0000","last_unstale":"2024-03-16T20:45:06.412143+0000","last_undegraded":"2024-03-16T20:45:06.412143+0000","last_fullsized":"2024-03-16T20:45:06.412143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:29:10.023600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000158+0000","last_change":"2024-03-16T20:45:05.414436+0000","last_active":"2024-03-16T20:45:07.000158+0000","last_peered":"2024-03-16T20:45:07.000158+0000","last_clean":"2024-03-16T20:45:07.000158+0000","last_became_active":"2024-03-16T20:45:05.413903+0000","last_became_peered":"2024-03-16T20:45:05.413903+0000","last_unstale":"2024-03-16T20:45:07.000158+0000","last_undegraded":"2024-03-16T20:45:07.000158+0000","last_fullsized":"2024-03-16T20:45:07.000158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:29:10.023600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413175+0000","last_change":"2024-03-16T20:45:05.423967+0000","last_active":"2024-03-16T20:45:06.413175+0000","last_peered":"2024-03-16T20:45:06.413175+0000","last_clean":"2024-03-16T20:45:06.413175+0000","last_became_active":"2024-03-16T20:45:05.423691+0000","last_became_peered":"2024-03-16T20:45:05.423691+0000","last_unstale":"2024-03-16T20:45:06.413175+0000","last_undegraded":"2024-03-16T20:45:06.413175+0000","last_fullsized":"2024-03-16T20:45:06.413175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:29:10.023600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410152+0000","last_change":"2024-03-16T20:45:05.418695+0000","last_active":"2024-03-16T20:45:06.410152+0000","last_peered":"2024-03-16T20:45:06.410152+0000","last_clean":"2024-03-16T20:45:06.410152+0000","last_became_active":"2024-03-16T20:45:05.415980+0000","last_became_peered":"2024-03-16T20:45:05.415980+0000","last_unstale":"2024-03-16T20:45:06.410152+0000","last_undegraded":"2024-03-16T20:45:06.410152+0000","last_fullsized":"2024-03-16T20:45:06.410152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:54:36.635249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000421+0000","last_change":"2024-03-16T20:45:05.414732+0000","last_active":"2024-03-16T20:45:07.000421+0000","last_peered":"2024-03-16T20:45:07.000421+0000","last_clean":"2024-03-16T20:45:07.000421+0000","last_became_active":"2024-03-16T20:45:05.411367+0000","last_became_peered":"2024-03-16T20:45:05.411367+0000","last_unstale":"2024-03-16T20:45:07.000421+0000","last_undegraded":"2024-03-16T20:45:07.000421+0000","last_fullsized":"2024-03-16T20:45:07.000421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:55:30.648219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337472+0000","last_change":"2024-03-16T20:45:05.422746+0000","last_active":"2024-03-16T20:45:07.337472+0000","last_peered":"2024-03-16T20:45:07.337472+0000","last_clean":"2024-03-16T20:45:07.337472+0000","last_became_active":"2024-03-16T20:45:05.422339+0000","last_became_peered":"2024-03-16T20:45:05.422339+0000","last_unstale":"2024-03-16T20:45:07.337472+0000","last_undegraded":"2024-03-16T20:45:07.337472+0000","last_fullsized":"2024-03-16T20:45:07.337472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:15:27.440101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413606+0000","last_change":"2024-03-16T20:45:05.419051+0000","last_active":"2024-03-16T20:45:06.413606+0000","last_peered":"2024-03-16T20:45:06.413606+0000","last_clean":"2024-03-16T20:45:06.413606+0000","last_became_active":"2024-03-16T20:45:05.417745+0000","last_became_peered":"2024-03-16T20:45:05.417745+0000","last_unstale":"2024-03-16T20:45:06.413606+0000","last_undegraded":"2024-03-16T20:45:06.413606+0000","last_fullsized":"2024-03-16T20:45:06.413606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:15:41.403418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.340377+0000","last_change":"2024-03-16T20:45:05.409514+0000","last_active":"2024-03-16T20:45:07.340377+0000","last_peered":"2024-03-16T20:45:07.340377+0000","last_clean":"2024-03-16T20:45:07.340377+0000","last_became_active":"2024-03-16T20:45:05.408695+0000","last_became_peered":"2024-03-16T20:45:05.408695+0000","last_unstale":"2024-03-16T20:45:07.340377+0000","last_undegraded":"2024-03-16T20:45:07.340377+0000","last_fullsized":"2024-03-16T20:45:07.340377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:31:58.827839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.411788+0000","last_change":"2024-03-16T20:45:05.417983+0000","last_active":"2024-03-16T20:45:06.411788+0000","last_peered":"2024-03-16T20:45:06.411788+0000","last_clean":"2024-03-16T20:45:06.411788+0000","last_became_active":"2024-03-16T20:45:05.417669+0000","last_became_peered":"2024-03-16T20:45:05.417669+0000","last_unstale":"2024-03-16T20:45:06.411788+0000","last_undegraded":"2024-03-16T20:45:06.411788+0000","last_fullsized":"2024-03-16T20:45:06.411788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:15:27.440101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.336150+0000","last_change":"2024-03-16T20:45:05.407932+0000","last_active":"2024-03-16T20:45:07.336150+0000","last_peered":"2024-03-16T20:45:07.336150+0000","last_clean":"2024-03-16T20:45:07.336150+0000","last_became_active":"2024-03-16T20:45:05.407504+0000","last_became_peered":"2024-03-16T20:45:05.407504+0000","last_unstale":"2024-03-16T20:45:07.336150+0000","last_undegraded":"2024-03-16T20:45:07.336150+0000","last_fullsized":"2024-03-16T20:45:07.336150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:23:33.686287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000584+0000","last_change":"2024-03-16T20:45:04.380160+0000","last_active":"2024-03-16T20:45:07.000584+0000","last_peered":"2024-03-16T20:45:07.000584+0000","last_clean":"2024-03-16T20:45:07.000584+0000","last_became_active":"2024-03-16T20:45:04.379622+0000","last_became_peered":"2024-03-16T20:45:04.379622+0000","last_unstale":"2024-03-16T20:45:07.000584+0000","last_undegraded":"2024-03-16T20:45:07.000584+0000","last_fullsized":"2024-03-16T20:45:07.000584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:57.399645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337841+0000","last_change":"2024-03-16T20:45:05.421374+0000","last_active":"2024-03-16T20:45:07.337841+0000","last_peered":"2024-03-16T20:45:07.337841+0000","last_clean":"2024-03-16T20:45:07.337841+0000","last_became_active":"2024-03-16T20:45:05.420935+0000","last_became_peered":"2024-03-16T20:45:05.420935+0000","last_unstale":"2024-03-16T20:45:07.337841+0000","last_undegraded":"2024-03-16T20:45:07.337841+0000","last_fullsized":"2024-03-16T20:45:07.337841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:26:51.023909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412742+0000","last_change":"2024-03-16T20:45:04.389177+0000","last_active":"2024-03-16T20:45:06.412742+0000","last_peered":"2024-03-16T20:45:06.412742+0000","last_clean":"2024-03-16T20:45:06.412742+0000","last_became_active":"2024-03-16T20:45:04.388433+0000","last_became_peered":"2024-03-16T20:45:04.388433+0000","last_unstale":"2024-03-16T20:45:06.412742+0000","last_undegraded":"2024-03-16T20:45:06.412742+0000","last_fullsized":"2024-03-16T20:45:06.412742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:48:48.175266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.999581+0000","last_change":"2024-03-16T20:45:04.383850+0000","last_active":"2024-03-16T20:45:06.999581+0000","last_peered":"2024-03-16T20:45:06.999581+0000","last_clean":"2024-03-16T20:45:06.999581+0000","last_became_active":"2024-03-16T20:45:04.383588+0000","last_became_peered":"2024-03-16T20:45:04.383588+0000","last_unstale":"2024-03-16T20:45:06.999581+0000","last_undegraded":"2024-03-16T20:45:06.999581+0000","last_fullsized":"2024-03-16T20:45:06.999581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:50:16.547893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.999693+0000","last_change":"2024-03-16T20:45:05.414605+0000","last_active":"2024-03-16T20:45:06.999693+0000","last_peered":"2024-03-16T20:45:06.999693+0000","last_clean":"2024-03-16T20:45:06.999693+0000","last_became_active":"2024-03-16T20:45:05.414067+0000","last_became_peered":"2024-03-16T20:45:05.414067+0000","last_unstale":"2024-03-16T20:45:06.999693+0000","last_undegraded":"2024-03-16T20:45:06.999693+0000","last_fullsized":"2024-03-16T20:45:06.999693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:26:51.023909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412395+0000","last_change":"2024-03-16T20:45:04.387317+0000","last_active":"2024-03-16T20:45:06.412395+0000","last_peered":"2024-03-16T20:45:06.412395+0000","last_clean":"2024-03-16T20:45:06.412395+0000","last_became_active":"2024-03-16T20:45:04.386147+0000","last_became_peered":"2024-03-16T20:45:04.386147+0000","last_unstale":"2024-03-16T20:45:06.412395+0000","last_undegraded":"2024-03-16T20:45:06.412395+0000","last_fullsized":"2024-03-16T20:45:06.412395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:48:48.175266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410049+0000","last_change":"2024-03-16T20:45:05.419053+0000","last_active":"2024-03-16T20:45:06.410049+0000","last_peered":"2024-03-16T20:45:06.410049+0000","last_clean":"2024-03-16T20:45:06.410049+0000","last_became_active":"2024-03-16T20:45:05.418610+0000","last_became_peered":"2024-03-16T20:45:05.418610+0000","last_unstale":"2024-03-16T20:45:06.410049+0000","last_undegraded":"2024-03-16T20:45:06.410049+0000","last_fullsized":"2024-03-16T20:45:06.410049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:03:04.229071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.414172+0000","last_change":"2024-03-16T20:45:04.400713+0000","last_active":"2024-03-16T20:45:06.414172+0000","last_peered":"2024-03-16T20:45:06.414172+0000","last_clean":"2024-03-16T20:45:06.414172+0000","last_became_active":"2024-03-16T20:45:04.399907+0000","last_became_peered":"2024-03-16T20:45:04.399907+0000","last_unstale":"2024-03-16T20:45:06.414172+0000","last_undegraded":"2024-03-16T20:45:06.414172+0000","last_fullsized":"2024-03-16T20:45:06.414172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:34:16.171892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413409+0000","last_change":"2024-03-16T20:45:05.422773+0000","last_active":"2024-03-16T20:45:06.413409+0000","last_peered":"2024-03-16T20:45:06.413409+0000","last_clean":"2024-03-16T20:45:06.413409+0000","last_became_active":"2024-03-16T20:45:05.421962+0000","last_became_peered":"2024-03-16T20:45:05.421962+0000","last_unstale":"2024-03-16T20:45:06.413409+0000","last_undegraded":"2024-03-16T20:45:06.413409+0000","last_fullsized":"2024-03-16T20:45:06.413409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:11:19.673375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413369+0000","last_change":"2024-03-16T20:45:04.395629+0000","last_active":"2024-03-16T20:45:06.413369+0000","last_peered":"2024-03-16T20:45:06.413369+0000","last_clean":"2024-03-16T20:45:06.413369+0000","last_became_active":"2024-03-16T20:45:04.395107+0000","last_became_peered":"2024-03-16T20:45:04.395107+0000","last_unstale":"2024-03-16T20:45:06.413369+0000","last_undegraded":"2024-03-16T20:45:06.413369+0000","last_fullsized":"2024-03-16T20:45:06.413369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:03:03.237004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.339518+0000","last_change":"2024-03-16T20:45:05.421014+0000","last_active":"2024-03-16T20:45:07.339518+0000","last_peered":"2024-03-16T20:45:07.339518+0000","last_clean":"2024-03-16T20:45:07.339518+0000","last_became_active":"2024-03-16T20:45:05.420652+0000","last_became_peered":"2024-03-16T20:45:05.420652+0000","last_unstale":"2024-03-16T20:45:07.339518+0000","last_undegraded":"2024-03-16T20:45:07.339518+0000","last_fullsized":"2024-03-16T20:45:07.339518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:35:07.902704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410232+0000","last_change":"2024-03-16T20:45:04.396880+0000","last_active":"2024-03-16T20:45:06.410232+0000","last_peered":"2024-03-16T20:45:06.410232+0000","last_clean":"2024-03-16T20:45:06.410232+0000","last_became_active":"2024-03-16T20:45:04.396654+0000","last_became_peered":"2024-03-16T20:45:04.396654+0000","last_unstale":"2024-03-16T20:45:06.410232+0000","last_undegraded":"2024-03-16T20:45:06.410232+0000","last_fullsized":"2024-03-16T20:45:06.410232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:03:03.237004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.336336+0000","last_change":"2024-03-16T20:45:05.425535+0000","last_active":"2024-03-16T20:45:07.336336+0000","last_peered":"2024-03-16T20:45:07.336336+0000","last_clean":"2024-03-16T20:45:07.336336+0000","last_became_active":"2024-03-16T20:45:05.425236+0000","last_became_peered":"2024-03-16T20:45:05.425236+0000","last_unstale":"2024-03-16T20:45:07.336336+0000","last_undegraded":"2024-03-16T20:45:07.336336+0000","last_fullsized":"2024-03-16T20:45:07.336336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:48:49.183385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412682+0000","last_change":"2024-03-16T20:45:04.384514+0000","last_active":"2024-03-16T20:45:06.412682+0000","last_peered":"2024-03-16T20:45:06.412682+0000","last_clean":"2024-03-16T20:45:06.412682+0000","last_became_active":"2024-03-16T20:45:04.379541+0000","last_became_peered":"2024-03-16T20:45:04.379541+0000","last_unstale":"2024-03-16T20:45:06.412682+0000","last_undegraded":"2024-03-16T20:45:06.412682+0000","last_fullsized":"2024-03-16T20:45:06.412682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:28:48.672077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459401+0000","last_change":"2024-03-16T20:45:05.416547+0000","last_active":"2024-03-16T20:45:06.459401+0000","last_peered":"2024-03-16T20:45:06.459401+0000","last_clean":"2024-03-16T20:45:06.459401+0000","last_became_active":"2024-03-16T20:45:05.416216+0000","last_became_peered":"2024-03-16T20:45:05.416216+0000","last_unstale":"2024-03-16T20:45:06.459401+0000","last_undegraded":"2024-03-16T20:45:06.459401+0000","last_fullsized":"2024-03-16T20:45:06.459401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:11:19.673375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.338023+0000","last_change":"2024-03-16T20:45:04.378488+0000","last_active":"2024-03-16T20:45:07.338023+0000","last_peered":"2024-03-16T20:45:07.338023+0000","last_clean":"2024-03-16T20:45:07.338023+0000","last_became_active":"2024-03-16T20:45:04.377819+0000","last_became_peered":"2024-03-16T20:45:04.377819+0000","last_unstale":"2024-03-16T20:45:07.338023+0000","last_undegraded":"2024-03-16T20:45:07.338023+0000","last_fullsized":"2024-03-16T20:45:07.338023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:09:39.701069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337814+0000","last_change":"2024-03-16T20:45:05.421867+0000","last_active":"2024-03-16T20:45:07.337814+0000","last_peered":"2024-03-16T20:45:07.337814+0000","last_clean":"2024-03-16T20:45:07.337814+0000","last_became_active":"2024-03-16T20:45:05.421377+0000","last_became_peered":"2024-03-16T20:45:05.421377+0000","last_unstale":"2024-03-16T20:45:07.337814+0000","last_undegraded":"2024-03-16T20:45:07.337814+0000","last_fullsized":"2024-03-16T20:45:07.337814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:03:04.229071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413633+0000","last_change":"2024-03-16T20:45:04.401205+0000","last_active":"2024-03-16T20:45:06.413633+0000","last_peered":"2024-03-16T20:45:06.413633+0000","last_clean":"2024-03-16T20:45:06.413633+0000","last_became_active":"2024-03-16T20:45:04.396416+0000","last_became_peered":"2024-03-16T20:45:04.396416+0000","last_unstale":"2024-03-16T20:45:06.413633+0000","last_undegraded":"2024-03-16T20:45:06.413633+0000","last_fullsized":"2024-03-16T20:45:06.413633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:28:48.672077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459532+0000","last_change":"2024-03-16T20:45:05.421976+0000","last_active":"2024-03-16T20:45:06.459532+0000","last_peered":"2024-03-16T20:45:06.459532+0000","last_clean":"2024-03-16T20:45:06.459532+0000","last_became_active":"2024-03-16T20:45:05.421370+0000","last_became_peered":"2024-03-16T20:45:05.421370+0000","last_unstale":"2024-03-16T20:45:06.459532+0000","last_undegraded":"2024-03-16T20:45:06.459532+0000","last_fullsized":"2024-03-16T20:45:06.459532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:15:41.403418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.411884+0000","last_change":"2024-03-16T20:45:04.389332+0000","last_active":"2024-03-16T20:45:06.411884+0000","last_peered":"2024-03-16T20:45:06.411884+0000","last_clean":"2024-03-16T20:45:06.411884+0000","last_became_active":"2024-03-16T20:45:04.388661+0000","last_became_peered":"2024-03-16T20:45:04.388661+0000","last_unstale":"2024-03-16T20:45:06.411884+0000","last_undegraded":"2024-03-16T20:45:06.411884+0000","last_fullsized":"2024-03-16T20:45:06.411884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:23:32.678168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412210+0000","last_change":"2024-03-16T20:45:05.418621+0000","last_active":"2024-03-16T20:45:06.412210+0000","last_peered":"2024-03-16T20:45:06.412210+0000","last_clean":"2024-03-16T20:45:06.412210+0000","last_became_active":"2024-03-16T20:45:05.418292+0000","last_became_peered":"2024-03-16T20:45:05.418292+0000","last_unstale":"2024-03-16T20:45:06.412210+0000","last_undegraded":"2024-03-16T20:45:06.412210+0000","last_fullsized":"2024-03-16T20:45:06.412210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:22:42.961272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337037+0000","last_change":"2024-03-16T20:45:04.359171+0000","last_active":"2024-03-16T20:45:07.337037+0000","last_peered":"2024-03-16T20:45:07.337037+0000","last_clean":"2024-03-16T20:45:07.337037+0000","last_became_active":"2024-03-16T20:45:04.358863+0000","last_became_peered":"2024-03-16T20:45:04.358863+0000","last_unstale":"2024-03-16T20:45:07.337037+0000","last_undegraded":"2024-03-16T20:45:07.337037+0000","last_fullsized":"2024-03-16T20:45:07.337037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:28:48.672077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459047+0000","last_change":"2024-03-16T20:45:05.417061+0000","last_active":"2024-03-16T20:45:06.459047+0000","last_peered":"2024-03-16T20:45:06.459047+0000","last_clean":"2024-03-16T20:45:06.459047+0000","last_became_active":"2024-03-16T20:45:05.416454+0000","last_became_peered":"2024-03-16T20:45:05.416454+0000","last_unstale":"2024-03-16T20:45:06.459047+0000","last_undegraded":"2024-03-16T20:45:06.459047+0000","last_fullsized":"2024-03-16T20:45:06.459047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:55:30.648219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.336372+0000","last_change":"2024-03-16T20:45:04.376055+0000","last_active":"2024-03-16T20:45:07.336372+0000","last_peered":"2024-03-16T20:45:07.336372+0000","last_clean":"2024-03-16T20:45:07.336372+0000","last_became_active":"2024-03-16T20:45:04.375272+0000","last_became_peered":"2024-03-16T20:45:04.375272+0000","last_unstale":"2024-03-16T20:45:07.336372+0000","last_undegraded":"2024-03-16T20:45:07.336372+0000","last_fullsized":"2024-03-16T20:45:07.336372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:52:31.895539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459647+0000","last_change":"2024-03-16T20:45:05.420665+0000","last_active":"2024-03-16T20:45:06.459647+0000","last_peered":"2024-03-16T20:45:06.459647+0000","last_clean":"2024-03-16T20:45:06.459647+0000","last_became_active":"2024-03-16T20:45:05.420363+0000","last_became_peered":"2024-03-16T20:45:05.420363+0000","last_unstale":"2024-03-16T20:45:06.459647+0000","last_undegraded":"2024-03-16T20:45:06.459647+0000","last_fullsized":"2024-03-16T20:45:06.459647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:59:02.640185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337985+0000","last_change":"2024-03-16T20:45:04.363877+0000","last_active":"2024-03-16T20:45:07.337985+0000","last_peered":"2024-03-16T20:45:07.337985+0000","last_clean":"2024-03-16T20:45:07.337985+0000","last_became_active":"2024-03-16T20:45:04.363180+0000","last_became_peered":"2024-03-16T20:45:04.363180+0000","last_unstale":"2024-03-16T20:45:07.337985+0000","last_undegraded":"2024-03-16T20:45:07.337985+0000","last_fullsized":"2024-03-16T20:45:07.337985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:57.399645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.460378+0000","last_change":"2024-03-16T20:45:05.429416+0000","last_active":"2024-03-16T20:45:06.460378+0000","last_peered":"2024-03-16T20:45:06.460378+0000","last_clean":"2024-03-16T20:45:06.460378+0000","last_became_active":"2024-03-16T20:45:05.428766+0000","last_became_peered":"2024-03-16T20:45:05.428766+0000","last_unstale":"2024-03-16T20:45:06.460378+0000","last_undegraded":"2024-03-16T20:45:06.460378+0000","last_fullsized":"2024-03-16T20:45:06.460378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:45:30.989735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410331+0000","last_change":"2024-03-16T20:45:04.355156+0000","last_active":"2024-03-16T20:45:06.410331+0000","last_peered":"2024-03-16T20:45:06.410331+0000","last_clean":"2024-03-16T20:45:06.410331+0000","last_became_active":"2024-03-16T20:45:04.354621+0000","last_became_peered":"2024-03-16T20:45:04.354621+0000","last_unstale":"2024-03-16T20:45:06.410331+0000","last_undegraded":"2024-03-16T20:45:06.410331+0000","last_fullsized":"2024-03-16T20:45:06.410331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:31:57.819721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413605+0000","last_change":"2024-03-16T20:45:05.419655+0000","last_active":"2024-03-16T20:45:06.413605+0000","last_peered":"2024-03-16T20:45:06.413605+0000","last_clean":"2024-03-16T20:45:06.413605+0000","last_became_active":"2024-03-16T20:45:05.417669+0000","last_became_peered":"2024-03-16T20:45:05.417669+0000","last_unstale":"2024-03-16T20:45:06.413605+0000","last_undegraded":"2024-03-16T20:45:06.413605+0000","last_fullsized":"2024-03-16T20:45:06.413605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:59:02.640185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.338415+0000","last_change":"2024-03-16T20:45:04.401781+0000","last_active":"2024-03-16T20:45:07.338415+0000","last_peered":"2024-03-16T20:45:07.338415+0000","last_clean":"2024-03-16T20:45:07.338415+0000","last_became_active":"2024-03-16T20:45:04.401563+0000","last_became_peered":"2024-03-16T20:45:04.401563+0000","last_unstale":"2024-03-16T20:45:07.338415+0000","last_undegraded":"2024-03-16T20:45:07.338415+0000","last_fullsized":"2024-03-16T20:45:07.338415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:34:16.171892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459736+0000","last_change":"2024-03-16T20:45:05.413949+0000","last_active":"2024-03-16T20:45:06.459736+0000","last_peered":"2024-03-16T20:45:06.459736+0000","last_clean":"2024-03-16T20:45:06.459736+0000","last_became_active":"2024-03-16T20:45:05.413609+0000","last_became_peered":"2024-03-16T20:45:05.413609+0000","last_unstale":"2024-03-16T20:45:06.459736+0000","last_undegraded":"2024-03-16T20:45:06.459736+0000","last_fullsized":"2024-03-16T20:45:06.459736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:18:48.237612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.458981+0000","last_change":"2024-03-16T20:44:54.293418+0000","last_active":"2024-03-16T20:45:06.458981+0000","last_peered":"2024-03-16T20:45:06.458981+0000","last_clean":"2024-03-16T20:45:06.458981+0000","last_became_active":"2024-03-16T20:44:54.291347+0000","last_became_peered":"2024-03-16T20:44:54.291347+0000","last_unstale":"2024-03-16T20:45:06.458981+0000","last_undegraded":"2024-03-16T20:45:06.458981+0000","last_fullsized":"2024-03-16T20:45:06.458981+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_clean_scrub_stamp":"2024-03-16T20:44:53.077639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:29:18.318494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412202+0000","last_change":"2024-03-16T20:45:04.394662+0000","last_active":"2024-03-16T20:45:06.412202+0000","last_peered":"2024-03-16T20:45:06.412202+0000","last_clean":"2024-03-16T20:45:06.412202+0000","last_became_active":"2024-03-16T20:45:04.394119+0000","last_became_peered":"2024-03-16T20:45:04.394119+0000","last_unstale":"2024-03-16T20:45:06.412202+0000","last_undegraded":"2024-03-16T20:45:06.412202+0000","last_fullsized":"2024-03-16T20:45:06.412202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:34:16.171892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000045+0000","last_change":"2024-03-16T20:45:05.413913+0000","last_active":"2024-03-16T20:45:07.000045+0000","last_peered":"2024-03-16T20:45:07.000045+0000","last_clean":"2024-03-16T20:45:07.000045+0000","last_became_active":"2024-03-16T20:45:05.412780+0000","last_became_peered":"2024-03-16T20:45:05.412780+0000","last_unstale":"2024-03-16T20:45:07.000045+0000","last_undegraded":"2024-03-16T20:45:07.000045+0000","last_fullsized":"2024-03-16T20:45:07.000045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:54:36.635249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.458865+0000","last_change":"2024-03-16T20:45:04.380237+0000","last_active":"2024-03-16T20:45:06.458865+0000","last_peered":"2024-03-16T20:45:06.458865+0000","last_clean":"2024-03-16T20:45:06.458865+0000","last_became_active":"2024-03-16T20:45:04.379086+0000","last_became_peered":"2024-03-16T20:45:04.379086+0000","last_unstale":"2024-03-16T20:45:06.458865+0000","last_undegraded":"2024-03-16T20:45:06.458865+0000","last_fullsized":"2024-03-16T20:45:06.458865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:23:32.678168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.336799+0000","last_change":"2024-03-16T20:45:05.414361+0000","last_active":"2024-03-16T20:45:07.336799+0000","last_peered":"2024-03-16T20:45:07.336799+0000","last_clean":"2024-03-16T20:45:07.336799+0000","last_became_active":"2024-03-16T20:45:05.411900+0000","last_became_peered":"2024-03-16T20:45:05.411900+0000","last_unstale":"2024-03-16T20:45:07.336799+0000","last_undegraded":"2024-03-16T20:45:07.336799+0000","last_fullsized":"2024-03-16T20:45:07.336799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:16:59.281226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.340530+0000","last_change":"2024-03-16T20:45:04.363289+0000","last_active":"2024-03-16T20:45:07.340530+0000","last_peered":"2024-03-16T20:45:07.340530+0000","last_clean":"2024-03-16T20:45:07.340530+0000","last_became_active":"2024-03-16T20:45:04.362919+0000","last_became_peered":"2024-03-16T20:45:04.362919+0000","last_unstale":"2024-03-16T20:45:07.340530+0000","last_undegraded":"2024-03-16T20:45:07.340530+0000","last_fullsized":"2024-03-16T20:45:07.340530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:50:16.547893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.409757+0000","last_change":"2024-03-16T20:45:05.408984+0000","last_active":"2024-03-16T20:45:06.409757+0000","last_peered":"2024-03-16T20:45:06.409757+0000","last_clean":"2024-03-16T20:45:06.409757+0000","last_became_active":"2024-03-16T20:45:05.408648+0000","last_became_peered":"2024-03-16T20:45:05.408648+0000","last_unstale":"2024-03-16T20:45:06.409757+0000","last_undegraded":"2024-03-16T20:45:06.409757+0000","last_fullsized":"2024-03-16T20:45:06.409757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:22:42.961272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413903+0000","last_change":"2024-03-16T20:45:04.394583+0000","last_active":"2024-03-16T20:45:06.413903+0000","last_peered":"2024-03-16T20:45:06.413903+0000","last_clean":"2024-03-16T20:45:06.413903+0000","last_became_active":"2024-03-16T20:45:04.394016+0000","last_became_peered":"2024-03-16T20:45:04.394016+0000","last_unstale":"2024-03-16T20:45:06.413903+0000","last_undegraded":"2024-03-16T20:45:06.413903+0000","last_fullsized":"2024-03-16T20:45:06.413903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:23:32.678168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.336052+0000","last_change":"2024-03-16T20:45:05.425183+0000","last_active":"2024-03-16T20:45:07.336052+0000","last_peered":"2024-03-16T20:45:07.336052+0000","last_clean":"2024-03-16T20:45:07.336052+0000","last_became_active":"2024-03-16T20:45:05.424862+0000","last_became_peered":"2024-03-16T20:45:05.424862+0000","last_unstale":"2024-03-16T20:45:07.336052+0000","last_undegraded":"2024-03-16T20:45:07.336052+0000","last_fullsized":"2024-03-16T20:45:07.336052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:03:04.245122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410481+0000","last_change":"2024-03-16T20:45:04.354012+0000","last_active":"2024-03-16T20:45:06.410481+0000","last_peered":"2024-03-16T20:45:06.410481+0000","last_clean":"2024-03-16T20:45:06.410481+0000","last_became_active":"2024-03-16T20:45:04.353589+0000","last_became_peered":"2024-03-16T20:45:04.353589+0000","last_unstale":"2024-03-16T20:45:06.410481+0000","last_undegraded":"2024-03-16T20:45:06.410481+0000","last_fullsized":"2024-03-16T20:45:06.410481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:48:48.175266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.415027+0000","last_change":"2024-03-16T20:45:05.417774+0000","last_active":"2024-03-16T20:45:06.415027+0000","last_peered":"2024-03-16T20:45:06.415027+0000","last_clean":"2024-03-16T20:45:06.415027+0000","last_became_active":"2024-03-16T20:45:05.417404+0000","last_became_peered":"2024-03-16T20:45:05.417404+0000","last_unstale":"2024-03-16T20:45:06.415027+0000","last_undegraded":"2024-03-16T20:45:06.415027+0000","last_fullsized":"2024-03-16T20:45:06.415027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:57:45.943288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459976+0000","last_change":"2024-03-16T20:45:04.381406+0000","last_active":"2024-03-16T20:45:06.459976+0000","last_peered":"2024-03-16T20:45:06.459976+0000","last_clean":"2024-03-16T20:45:06.459976+0000","last_became_active":"2024-03-16T20:45:04.380608+0000","last_became_peered":"2024-03-16T20:45:04.380608+0000","last_unstale":"2024-03-16T20:45:06.459976+0000","last_undegraded":"2024-03-16T20:45:06.459976+0000","last_fullsized":"2024-03-16T20:45:06.459976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:16:39.193904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410306+0000","last_change":"2024-03-16T20:45:05.408949+0000","last_active":"2024-03-16T20:45:06.410306+0000","last_peered":"2024-03-16T20:45:06.410306+0000","last_clean":"2024-03-16T20:45:06.410306+0000","last_became_active":"2024-03-16T20:45:05.408619+0000","last_became_peered":"2024-03-16T20:45:05.408619+0000","last_unstale":"2024-03-16T20:45:06.410306+0000","last_undegraded":"2024-03-16T20:45:06.410306+0000","last_fullsized":"2024-03-16T20:45:06.410306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:35:07.902704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.611129+0000","last_change":"2024-03-16T20:45:04.383772+0000","last_active":"2024-03-16T20:45:06.611129+0000","last_peered":"2024-03-16T20:45:06.611129+0000","last_clean":"2024-03-16T20:45:06.611129+0000","last_became_active":"2024-03-16T20:45:04.383050+0000","last_became_peered":"2024-03-16T20:45:04.383050+0000","last_unstale":"2024-03-16T20:45:06.611129+0000","last_undegraded":"2024-03-16T20:45:06.611129+0000","last_fullsized":"2024-03-16T20:45:06.611129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:20:08.814290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000319+0000","last_change":"2024-03-16T20:45:05.419648+0000","last_active":"2024-03-16T20:45:07.000319+0000","last_peered":"2024-03-16T20:45:07.000319+0000","last_clean":"2024-03-16T20:45:07.000319+0000","last_became_active":"2024-03-16T20:45:05.419358+0000","last_became_peered":"2024-03-16T20:45:05.419358+0000","last_unstale":"2024-03-16T20:45:07.000319+0000","last_undegraded":"2024-03-16T20:45:07.000319+0000","last_fullsized":"2024-03-16T20:45:07.000319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:50:22.920748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.414177+0000","last_change":"2024-03-16T20:45:04.382826+0000","last_active":"2024-03-16T20:45:06.414177+0000","last_peered":"2024-03-16T20:45:06.414177+0000","last_clean":"2024-03-16T20:45:06.414177+0000","last_became_active":"2024-03-16T20:45:04.381824+0000","last_became_peered":"2024-03-16T20:45:04.381824+0000","last_unstale":"2024-03-16T20:45:06.414177+0000","last_undegraded":"2024-03-16T20:45:06.414177+0000","last_fullsized":"2024-03-16T20:45:06.414177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:03:03.237004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337732+0000","last_change":"2024-03-16T20:45:05.415644+0000","last_active":"2024-03-16T20:45:07.337732+0000","last_peered":"2024-03-16T20:45:07.337732+0000","last_clean":"2024-03-16T20:45:07.337732+0000","last_became_active":"2024-03-16T20:45:05.415294+0000","last_became_peered":"2024-03-16T20:45:05.415294+0000","last_unstale":"2024-03-16T20:45:07.337732+0000","last_undegraded":"2024-03-16T20:45:07.337732+0000","last_fullsized":"2024-03-16T20:45:07.337732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:22:42.961272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.458835+0000","last_change":"2024-03-16T20:45:04.380383+0000","last_active":"2024-03-16T20:45:06.458835+0000","last_peered":"2024-03-16T20:45:06.458835+0000","last_clean":"2024-03-16T20:45:06.458835+0000","last_became_active":"2024-03-16T20:45:04.379121+0000","last_became_peered":"2024-03-16T20:45:04.379121+0000","last_unstale":"2024-03-16T20:45:06.458835+0000","last_undegraded":"2024-03-16T20:45:06.458835+0000","last_fullsized":"2024-03-16T20:45:06.458835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:03:03.237004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.336388+0000","last_change":"2024-03-16T20:45:05.408820+0000","last_active":"2024-03-16T20:45:07.336388+0000","last_peered":"2024-03-16T20:45:07.336388+0000","last_clean":"2024-03-16T20:45:07.336388+0000","last_became_active":"2024-03-16T20:45:05.408337+0000","last_became_peered":"2024-03-16T20:45:05.408337+0000","last_unstale":"2024-03-16T20:45:07.336388+0000","last_undegraded":"2024-03-16T20:45:07.336388+0000","last_fullsized":"2024-03-16T20:45:07.336388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:31:58.827839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410303+0000","last_change":"2024-03-16T20:45:04.379953+0000","last_active":"2024-03-16T20:45:06.410303+0000","last_peered":"2024-03-16T20:45:06.410303+0000","last_clean":"2024-03-16T20:45:06.410303+0000","last_became_active":"2024-03-16T20:45:04.379610+0000","last_became_peered":"2024-03-16T20:45:04.379610+0000","last_unstale":"2024-03-16T20:45:06.410303+0000","last_undegraded":"2024-03-16T20:45:06.410303+0000","last_fullsized":"2024-03-16T20:45:06.410303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:23:32.678168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000614+0000","last_change":"2024-03-16T20:45:05.401685+0000","last_active":"2024-03-16T20:45:07.000614+0000","last_peered":"2024-03-16T20:45:07.000614+0000","last_clean":"2024-03-16T20:45:07.000614+0000","last_became_active":"2024-03-16T20:45:05.399878+0000","last_became_peered":"2024-03-16T20:45:05.399878+0000","last_unstale":"2024-03-16T20:45:07.000614+0000","last_undegraded":"2024-03-16T20:45:07.000614+0000","last_fullsized":"2024-03-16T20:45:07.000614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:11:19.673375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410121+0000","last_change":"2024-03-16T20:45:04.359852+0000","last_active":"2024-03-16T20:45:06.410121+0000","last_peered":"2024-03-16T20:45:06.410121+0000","last_clean":"2024-03-16T20:45:06.410121+0000","last_became_active":"2024-03-16T20:45:04.359557+0000","last_became_peered":"2024-03-16T20:45:04.359557+0000","last_unstale":"2024-03-16T20:45:06.410121+0000","last_undegraded":"2024-03-16T20:45:06.410121+0000","last_fullsized":"2024-03-16T20:45:06.410121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:34:16.171892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000547+0000","last_change":"2024-03-16T20:45:05.422880+0000","last_active":"2024-03-16T20:45:07.000547+0000","last_peered":"2024-03-16T20:45:07.000547+0000","last_clean":"2024-03-16T20:45:07.000547+0000","last_became_active":"2024-03-16T20:45:05.422585+0000","last_became_peered":"2024-03-16T20:45:05.422585+0000","last_unstale":"2024-03-16T20:45:07.000547+0000","last_undegraded":"2024-03-16T20:45:07.000547+0000","last_fullsized":"2024-03-16T20:45:07.000547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:54.931097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459415+0000","last_change":"2024-03-16T20:45:04.382087+0000","last_active":"2024-03-16T20:45:06.459415+0000","last_peered":"2024-03-16T20:45:06.459415+0000","last_clean":"2024-03-16T20:45:06.459415+0000","last_became_active":"2024-03-16T20:45:04.381331+0000","last_became_peered":"2024-03-16T20:45:04.381331+0000","last_unstale":"2024-03-16T20:45:06.459415+0000","last_undegraded":"2024-03-16T20:45:06.459415+0000","last_fullsized":"2024-03-16T20:45:06.459415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:31:57.819721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412636+0000","last_change":"2024-03-16T20:45:05.416889+0000","last_active":"2024-03-16T20:45:06.412636+0000","last_peered":"2024-03-16T20:45:06.412636+0000","last_clean":"2024-03-16T20:45:06.412636+0000","last_became_active":"2024-03-16T20:45:05.416560+0000","last_became_peered":"2024-03-16T20:45:05.416560+0000","last_unstale":"2024-03-16T20:45:06.412636+0000","last_undegraded":"2024-03-16T20:45:06.412636+0000","last_fullsized":"2024-03-16T20:45:06.412636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:54:36.635249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.414635+0000","last_change":"2024-03-16T20:45:05.403182+0000","last_active":"2024-03-16T20:45:06.414635+0000","last_peered":"2024-03-16T20:45:06.414635+0000","last_clean":"2024-03-16T20:45:06.414635+0000","last_became_active":"2024-03-16T20:45:05.401656+0000","last_became_peered":"2024-03-16T20:45:05.401656+0000","last_unstale":"2024-03-16T20:45:06.414635+0000","last_undegraded":"2024-03-16T20:45:06.414635+0000","last_fullsized":"2024-03-16T20:45:06.414635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:40:17.773165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459476+0000","last_change":"2024-03-16T20:45:04.402308+0000","last_active":"2024-03-16T20:45:06.459476+0000","last_peered":"2024-03-16T20:45:06.459476+0000","last_clean":"2024-03-16T20:45:06.459476+0000","last_became_active":"2024-03-16T20:45:04.401750+0000","last_became_peered":"2024-03-16T20:45:04.401750+0000","last_unstale":"2024-03-16T20:45:06.459476+0000","last_undegraded":"2024-03-16T20:45:06.459476+0000","last_fullsized":"2024-03-16T20:45:06.459476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:48:48.175266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413317+0000","last_change":"2024-03-16T20:45:05.412664+0000","last_active":"2024-03-16T20:45:06.413317+0000","last_peered":"2024-03-16T20:45:06.413317+0000","last_clean":"2024-03-16T20:45:06.413317+0000","last_became_active":"2024-03-16T20:45:05.412302+0000","last_became_peered":"2024-03-16T20:45:05.412302+0000","last_unstale":"2024-03-16T20:45:06.413317+0000","last_undegraded":"2024-03-16T20:45:06.413317+0000","last_fullsized":"2024-03-16T20:45:06.413317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:55:30.648219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459690+0000","last_change":"2024-03-16T20:45:04.387378+0000","last_active":"2024-03-16T20:45:06.459690+0000","last_peered":"2024-03-16T20:45:06.459690+0000","last_clean":"2024-03-16T20:45:06.459690+0000","last_became_active":"2024-03-16T20:45:04.386602+0000","last_became_peered":"2024-03-16T20:45:04.386602+0000","last_unstale":"2024-03-16T20:45:06.459690+0000","last_undegraded":"2024-03-16T20:45:06.459690+0000","last_fullsized":"2024-03-16T20:45:06.459690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:20:08.814290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.339116+0000","last_change":"2024-03-16T20:45:05.413222+0000","last_active":"2024-03-16T20:45:07.339116+0000","last_peered":"2024-03-16T20:45:07.339116+0000","last_clean":"2024-03-16T20:45:07.339116+0000","last_became_active":"2024-03-16T20:45:05.412620+0000","last_became_peered":"2024-03-16T20:45:05.412620+0000","last_unstale":"2024-03-16T20:45:07.339116+0000","last_undegraded":"2024-03-16T20:45:07.339116+0000","last_fullsized":"2024-03-16T20:45:07.339116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:54:36.635249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000425+0000","last_change":"2024-03-16T20:45:04.362033+0000","last_active":"2024-03-16T20:45:07.000425+0000","last_peered":"2024-03-16T20:45:07.000425+0000","last_clean":"2024-03-16T20:45:07.000425+0000","last_became_active":"2024-03-16T20:45:04.361100+0000","last_became_peered":"2024-03-16T20:45:04.361100+0000","last_unstale":"2024-03-16T20:45:07.000425+0000","last_undegraded":"2024-03-16T20:45:07.000425+0000","last_fullsized":"2024-03-16T20:45:07.000425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:28:48.672077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000396+0000","last_change":"2024-03-16T20:45:05.425640+0000","last_active":"2024-03-16T20:45:07.000396+0000","last_peered":"2024-03-16T20:45:07.000396+0000","last_clean":"2024-03-16T20:45:07.000396+0000","last_became_active":"2024-03-16T20:45:05.424729+0000","last_became_peered":"2024-03-16T20:45:05.424729+0000","last_unstale":"2024-03-16T20:45:07.000396+0000","last_undegraded":"2024-03-16T20:45:07.000396+0000","last_fullsized":"2024-03-16T20:45:07.000396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:26:54.432519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.339770+0000","last_change":"2024-03-16T20:45:04.380444+0000","last_active":"2024-03-16T20:45:07.339770+0000","last_peered":"2024-03-16T20:45:07.339770+0000","last_clean":"2024-03-16T20:45:07.339770+0000","last_became_active":"2024-03-16T20:45:04.380164+0000","last_became_peered":"2024-03-16T20:45:04.380164+0000","last_unstale":"2024-03-16T20:45:07.339770+0000","last_undegraded":"2024-03-16T20:45:07.339770+0000","last_fullsized":"2024-03-16T20:45:07.339770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:43:46.634405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T20:45:12.561 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T20:45:12.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f761d181640 1 Processor -- start 2024-03-16T20:45:12.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f761d181640 1 -- start start 2024-03-16T20:45:12.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f761d181640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f76180baca0 0x7f7618144e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:12.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f761d181640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f761813f240 0x7f76180b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:12.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f761d181640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76180bb080 con 0x7f7618148ec0 2024-03-16T20:45:12.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f761d181640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76180b06b0 con 0x7f761813f240 2024-03-16T20:45:12.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f761d181640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76180b1cd0 con 0x7f76180baca0 2024-03-16T20:45:12.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f761759e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f761813f240 0x7f76180b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:12.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f761759e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f761813f240 0x7f76180b1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41620/0 (socket says 172.21.15.122:41620) 2024-03-16T20:45:12.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f761759e640 1 -- 172.21.15.122:0/2794846394 learned_addr learned my addr 172.21.15.122:0/2794846394 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:12.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f7616d9d640 1 --2- 172.21.15.122:0/2794846394 >> v2:172.21.15.195:3300/0 conn(0x7f76180baca0 0x7f7618144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:12.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f761759e640 1 -- 172.21.15.122:0/2794846394 >> v2:172.21.15.195:3300/0 conn(0x7f76180baca0 msgr2=0x7f7618144e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:12.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f761759e640 1 --2- 172.21.15.122:0/2794846394 >> v2:172.21.15.195:3300/0 conn(0x7f76180baca0 0x7f7618144e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f761759e640 1 -- 172.21.15.122:0/2794846394 >> v1:172.21.15.195:6789/0 conn(0x7f7618148ec0 legacy=0x7f761813eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f761759e640 1 -- 172.21.15.122:0/2794846394 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76180b1300 con 0x7f761813f240 2024-03-16T20:45:12.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f7616d9d640 1 --2- 172.21.15.122:0/2794846394 >> v2:172.21.15.195:3300/0 conn(0x7f76180baca0 0x7f7618144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:12.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f761759e640 1 --2- 172.21.15.122:0/2794846394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f761813f240 0x7f76180b1790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f76180bb790 tx=0x7f761001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a5782864522e82 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:12.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.712+0000 7f761659c640 1 -- 172.21.15.122:0/2794846394 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76100a5710 con 0x7f761813f240 2024-03-16T20:45:12.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761659c640 1 -- 172.21.15.122:0/2794846394 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76100a5870 con 0x7f761813f240 2024-03-16T20:45:12.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761659c640 1 -- 172.21.15.122:0/2794846394 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76100a5c00 con 0x7f761813f240 2024-03-16T20:45:12.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 -- 172.21.15.122:0/2794846394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f761813f240 msgr2=0x7f76180b1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 --2- 172.21.15.122:0/2794846394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f761813f240 0x7f76180b1790 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f76180bb790 tx=0x7f761001c960 comp rx=0 tx=0).stop 2024-03-16T20:45:12.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 -- 172.21.15.122:0/2794846394 shutdown_connections 2024-03-16T20:45:12.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 --2- 172.21.15.122:0/2794846394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f761813f240 0x7f76180b1790 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 --2- 172.21.15.122:0/2794846394 >> v2:172.21.15.195:3300/0 conn(0x7f76180baca0 0x7f7618144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 -- 172.21.15.122:0/2794846394 >> 172.21.15.122:0/2794846394 conn(0x7f761806ee40 msgr2=0x7f761806f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:12.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 -- 172.21.15.122:0/2794846394 shutdown_connections 2024-03-16T20:45:12.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 -- 172.21.15.122:0/2794846394 wait complete. 2024-03-16T20:45:12.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 Processor -- start 2024-03-16T20:45:12.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 -- start start 2024-03-16T20:45:12.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76180baca0 0x7f76180b30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:12.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f761813f240 0x7f7618141b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:12.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7618148ec0 0x7f7618142040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:12.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f761813fee0 con 0x7f76180baca0 2024-03-16T20:45:12.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7618148180 con 0x7f761813f240 2024-03-16T20:45:12.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76180b4330 con 0x7f7618148ec0 2024-03-16T20:45:12.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f7616d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76180baca0 0x7f76180b30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:12.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f7616d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76180baca0 0x7f76180b30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41636/0 (socket says 172.21.15.122:41636) 2024-03-16T20:45:12.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f7616d9d640 1 -- 172.21.15.122:0/2150866990 learned_addr learned my addr 172.21.15.122:0/2150866990 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f760ffff640 1 --2- 172.21.15.122:0/2150866990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f761813f240 0x7f7618141b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f7616d9d640 1 -- 172.21.15.122:0/2150866990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7618148ec0 msgr2=0x7f7618142040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f7616d9d640 1 --2- 172.21.15.122:0/2150866990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7618148ec0 0x7f7618142040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f7616d9d640 1 -- 172.21.15.122:0/2150866990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f761813f240 msgr2=0x7f7618141b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f7616d9d640 1 --2- 172.21.15.122:0/2150866990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f761813f240 0x7f7618141b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f7616d9d640 1 -- 172.21.15.122:0/2150866990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7610009b30 con 0x7f76180baca0 2024-03-16T20:45:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f760ffff640 1 --2- 172.21.15.122:0/2150866990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f761813f240 0x7f7618141b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f7616d9d640 1 --2- 172.21.15.122:0/2150866990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76180baca0 0x7f76180b30d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f760800e810 tx=0x7f760800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f760f7fe640 1 -- 172.21.15.122:0/2150866990 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760800c8d0 con 0x7f76180baca0 2024-03-16T20:45:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f760f7fe640 1 -- 172.21.15.122:0/2150866990 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760800ca30 con 0x7f76180baca0 2024-03-16T20:45:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 -- 172.21.15.122:0/2150866990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76181400e0 con 0x7f76180baca0 2024-03-16T20:45:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f760f7fe640 1 -- 172.21.15.122:0/2150866990 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f760800cdc0 con 0x7f76180baca0 2024-03-16T20:45:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 -- 172.21.15.122:0/2150866990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76180b4890 con 0x7f76180baca0 2024-03-16T20:45:12.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.716+0000 7f761d181640 1 -- 172.21.15.122:0/2150866990 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76180677b0 con 0x7f76180baca0 2024-03-16T20:45:12.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.720+0000 7f760f7fe640 1 -- 172.21.15.122:0/2150866990 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f7608013410 con 0x7f76180baca0 2024-03-16T20:45:12.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.720+0000 7f760f7fe640 1 --2- 172.21.15.122:0/2150866990 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f75f8066e40 0x7f75f8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:12.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.720+0000 7f760f7fe640 1 -- 172.21.15.122:0/2150866990 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76080847d0 con 0x7f76180baca0 2024-03-16T20:45:12.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.720+0000 7f760ffff640 1 --2- 172.21.15.122:0/2150866990 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f75f8066e40 0x7f75f8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:12.724 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.720+0000 7f760ffff640 1 --2- 172.21.15.122:0/2150866990 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f75f8066e40 0x7f75f8069300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f76180b1ed0 tx=0x7f7600009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:12.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.724+0000 7f760f7fe640 1 -- 172.21.15.122:0/2150866990 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7608051650 con 0x7f76180baca0 2024-03-16T20:45:12.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.864+0000 7f761d181640 1 -- 172.21.15.122:0/2150866990 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7618151fb0 con 0x7f75f8066e40 2024-03-16T20:45:12.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.884+0000 7f760f7fe640 1 -- 172.21.15.122:0/2150866990 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7f760000d4f0 con 0x7f75f8066e40 2024-03-16T20:45:12.888 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:12.891 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-03-16T20:45:12.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.888+0000 7f761d181640 1 -- 172.21.15.122:0/2150866990 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f75f8066e40 msgr2=0x7f75f8069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.888+0000 7f761d181640 1 --2- 172.21.15.122:0/2150866990 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f75f8066e40 0x7f75f8069300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f76180b1ed0 tx=0x7f7600009040 comp rx=0 tx=0).stop 2024-03-16T20:45:12.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.888+0000 7f761d181640 1 -- 172.21.15.122:0/2150866990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76180baca0 msgr2=0x7f76180b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:12.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.888+0000 7f761d181640 1 --2- 172.21.15.122:0/2150866990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76180baca0 0x7f76180b30d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f760800e810 tx=0x7f760800ecd0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.888+0000 7f761d181640 1 -- 172.21.15.122:0/2150866990 shutdown_connections 2024-03-16T20:45:12.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.888+0000 7f761d181640 1 --2- 172.21.15.122:0/2150866990 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f75f8066e40 0x7f75f8069300 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.888+0000 7f761d181640 1 --2- 172.21.15.122:0/2150866990 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7618148ec0 0x7f7618142040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.888+0000 7f761d181640 1 --2- 172.21.15.122:0/2150866990 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f761813f240 0x7f7618141b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.888+0000 7f761d181640 1 --2- 172.21.15.122:0/2150866990 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76180baca0 0x7f76180b30d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:12.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.888+0000 7f761d181640 1 -- 172.21.15.122:0/2150866990 >> 172.21.15.122:0/2150866990 conn(0x7f761806ee40 msgr2=0x7f7618143630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:12.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.888+0000 7f761d181640 1 -- 172.21.15.122:0/2150866990 shutdown_connections 2024-03-16T20:45:12.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:12.888+0000 7f761d181640 1 -- 172.21.15.122:0/2150866990 wait complete. 2024-03-16T20:45:12.906 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-03-16T20:45:12.579473+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235752,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738968,"statfs":{"total":773094113280,"available":772852703232,"internally_reserved":0,"allocated":2990080,"data_stored":1790396,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002615"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459997+0000","last_change":"2024-03-16T20:45:05.428065+0000","last_active":"2024-03-16T20:45:06.459997+0000","last_peered":"2024-03-16T20:45:06.459997+0000","last_clean":"2024-03-16T20:45:06.459997+0000","last_became_active":"2024-03-16T20:45:05.427498+0000","last_became_peered":"2024-03-16T20:45:05.427498+0000","last_unstale":"2024-03-16T20:45:06.459997+0000","last_undegraded":"2024-03-16T20:45:06.459997+0000","last_fullsized":"2024-03-16T20:45:06.459997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:40:17.773165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410546+0000","last_change":"2024-03-16T20:45:05.418257+0000","last_active":"2024-03-16T20:45:06.410546+0000","last_peered":"2024-03-16T20:45:06.410546+0000","last_clean":"2024-03-16T20:45:06.410546+0000","last_became_active":"2024-03-16T20:45:05.416660+0000","last_became_peered":"2024-03-16T20:45:05.416660+0000","last_unstale":"2024-03-16T20:45:06.410546+0000","last_undegraded":"2024-03-16T20:45:06.410546+0000","last_fullsized":"2024-03-16T20:45:06.410546+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:29:10.023600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337420+0000","last_change":"2024-03-16T20:45:05.410148+0000","last_active":"2024-03-16T20:45:07.337420+0000","last_peered":"2024-03-16T20:45:07.337420+0000","last_clean":"2024-03-16T20:45:07.337420+0000","last_became_active":"2024-03-16T20:45:05.409853+0000","last_became_peered":"2024-03-16T20:45:05.409853+0000","last_unstale":"2024-03-16T20:45:07.337420+0000","last_undegraded":"2024-03-16T20:45:07.337420+0000","last_fullsized":"2024-03-16T20:45:07.337420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:23:41.936524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412946+0000","last_change":"2024-03-16T20:45:05.419996+0000","last_active":"2024-03-16T20:45:06.412946+0000","last_peered":"2024-03-16T20:45:06.412946+0000","last_clean":"2024-03-16T20:45:06.412946+0000","last_became_active":"2024-03-16T20:45:05.417478+0000","last_became_peered":"2024-03-16T20:45:05.417478+0000","last_unstale":"2024-03-16T20:45:06.412946+0000","last_undegraded":"2024-03-16T20:45:06.412946+0000","last_fullsized":"2024-03-16T20:45:06.412946+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:26:54.432519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459267+0000","last_change":"2024-03-16T20:45:05.422236+0000","last_active":"2024-03-16T20:45:06.459267+0000","last_peered":"2024-03-16T20:45:06.459267+0000","last_clean":"2024-03-16T20:45:06.459267+0000","last_became_active":"2024-03-16T20:45:05.421698+0000","last_became_peered":"2024-03-16T20:45:05.421698+0000","last_unstale":"2024-03-16T20:45:06.459267+0000","last_undegraded":"2024-03-16T20:45:06.459267+0000","last_fullsized":"2024-03-16T20:45:06.459267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:54.931097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412907+0000","last_change":"2024-03-16T20:45:05.419865+0000","last_active":"2024-03-16T20:45:06.412907+0000","last_peered":"2024-03-16T20:45:06.412907+0000","last_clean":"2024-03-16T20:45:06.412907+0000","last_became_active":"2024-03-16T20:45:05.419476+0000","last_became_peered":"2024-03-16T20:45:05.419476+0000","last_unstale":"2024-03-16T20:45:06.412907+0000","last_undegraded":"2024-03-16T20:45:06.412907+0000","last_fullsized":"2024-03-16T20:45:06.412907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:50:22.920748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.458816+0000","last_change":"2024-03-16T20:45:05.421986+0000","last_active":"2024-03-16T20:45:06.458816+0000","last_peered":"2024-03-16T20:45:06.458816+0000","last_clean":"2024-03-16T20:45:06.458816+0000","last_became_active":"2024-03-16T20:45:05.421125+0000","last_became_peered":"2024-03-16T20:45:05.421125+0000","last_unstale":"2024-03-16T20:45:06.458816+0000","last_undegraded":"2024-03-16T20:45:06.458816+0000","last_fullsized":"2024-03-16T20:45:06.458816+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:26:54.432519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.999419+0000","last_change":"2024-03-16T20:45:05.402329+0000","last_active":"2024-03-16T20:45:06.999419+0000","last_peered":"2024-03-16T20:45:06.999419+0000","last_clean":"2024-03-16T20:45:06.999419+0000","last_became_active":"2024-03-16T20:45:05.400677+0000","last_became_peered":"2024-03-16T20:45:05.400677+0000","last_unstale":"2024-03-16T20:45:06.999419+0000","last_undegraded":"2024-03-16T20:45:06.999419+0000","last_fullsized":"2024-03-16T20:45:06.999419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:22:42.961272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.460167+0000","last_change":"2024-03-16T20:45:05.425819+0000","last_active":"2024-03-16T20:45:06.460167+0000","last_peered":"2024-03-16T20:45:06.460167+0000","last_clean":"2024-03-16T20:45:06.460167+0000","last_became_active":"2024-03-16T20:45:05.425238+0000","last_became_peered":"2024-03-16T20:45:05.425238+0000","last_unstale":"2024-03-16T20:45:06.460167+0000","last_undegraded":"2024-03-16T20:45:06.460167+0000","last_fullsized":"2024-03-16T20:45:06.460167+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:04:42.305935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412787+0000","last_change":"2024-03-16T20:45:05.419949+0000","last_active":"2024-03-16T20:45:06.412787+0000","last_peered":"2024-03-16T20:45:06.412787+0000","last_clean":"2024-03-16T20:45:06.412787+0000","last_became_active":"2024-03-16T20:45:05.419670+0000","last_became_peered":"2024-03-16T20:45:05.419670+0000","last_unstale":"2024-03-16T20:45:06.412787+0000","last_undegraded":"2024-03-16T20:45:06.412787+0000","last_fullsized":"2024-03-16T20:45:06.412787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:35:07.902704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412340+0000","last_change":"2024-03-16T20:45:05.424035+0000","last_active":"2024-03-16T20:45:06.412340+0000","last_peered":"2024-03-16T20:45:06.412340+0000","last_clean":"2024-03-16T20:45:06.412340+0000","last_became_active":"2024-03-16T20:45:05.423657+0000","last_became_peered":"2024-03-16T20:45:05.423657+0000","last_unstale":"2024-03-16T20:45:06.412340+0000","last_undegraded":"2024-03-16T20:45:06.412340+0000","last_fullsized":"2024-03-16T20:45:06.412340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:03:04.229071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.336271+0000","last_change":"2024-03-16T20:45:05.427007+0000","last_active":"2024-03-16T20:45:07.336271+0000","last_peered":"2024-03-16T20:45:07.336271+0000","last_clean":"2024-03-16T20:45:07.336271+0000","last_became_active":"2024-03-16T20:45:05.426482+0000","last_became_peered":"2024-03-16T20:45:05.426482+0000","last_unstale":"2024-03-16T20:45:07.336271+0000","last_undegraded":"2024-03-16T20:45:07.336271+0000","last_fullsized":"2024-03-16T20:45:07.336271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:20:09.822409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.999992+0000","last_change":"2024-03-16T20:45:05.418866+0000","last_active":"2024-03-16T20:45:06.999992+0000","last_peered":"2024-03-16T20:45:06.999992+0000","last_clean":"2024-03-16T20:45:06.999992+0000","last_became_active":"2024-03-16T20:45:05.418458+0000","last_became_peered":"2024-03-16T20:45:05.418458+0000","last_unstale":"2024-03-16T20:45:06.999992+0000","last_undegraded":"2024-03-16T20:45:06.999992+0000","last_fullsized":"2024-03-16T20:45:06.999992+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:35:07.902704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337384+0000","last_change":"2024-03-16T20:45:05.411615+0000","last_active":"2024-03-16T20:45:07.337384+0000","last_peered":"2024-03-16T20:45:07.337384+0000","last_clean":"2024-03-16T20:45:07.337384+0000","last_became_active":"2024-03-16T20:45:05.410789+0000","last_became_peered":"2024-03-16T20:45:05.410789+0000","last_unstale":"2024-03-16T20:45:07.337384+0000","last_undegraded":"2024-03-16T20:45:07.337384+0000","last_fullsized":"2024-03-16T20:45:07.337384+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:08:02.898870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410139+0000","last_change":"2024-03-16T20:45:05.411051+0000","last_active":"2024-03-16T20:45:06.410139+0000","last_peered":"2024-03-16T20:45:06.410139+0000","last_clean":"2024-03-16T20:45:06.410139+0000","last_became_active":"2024-03-16T20:45:05.409805+0000","last_became_peered":"2024-03-16T20:45:05.409805+0000","last_unstale":"2024-03-16T20:45:06.410139+0000","last_undegraded":"2024-03-16T20:45:06.410139+0000","last_fullsized":"2024-03-16T20:45:06.410139+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:26:51.023909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.411744+0000","last_change":"2024-03-16T20:45:05.413320+0000","last_active":"2024-03-16T20:45:06.411744+0000","last_peered":"2024-03-16T20:45:06.411744+0000","last_clean":"2024-03-16T20:45:06.411744+0000","last_became_active":"2024-03-16T20:45:05.412473+0000","last_became_peered":"2024-03-16T20:45:05.412473+0000","last_unstale":"2024-03-16T20:45:06.411744+0000","last_undegraded":"2024-03-16T20:45:06.411744+0000","last_fullsized":"2024-03-16T20:45:06.411744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:35:07.902704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.414480+0000","last_change":"2024-03-16T20:45:05.421924+0000","last_active":"2024-03-16T20:45:06.414480+0000","last_peered":"2024-03-16T20:45:06.414480+0000","last_clean":"2024-03-16T20:45:06.414480+0000","last_became_active":"2024-03-16T20:45:05.421632+0000","last_became_peered":"2024-03-16T20:45:05.421632+0000","last_unstale":"2024-03-16T20:45:06.414480+0000","last_undegraded":"2024-03-16T20:45:06.414480+0000","last_fullsized":"2024-03-16T20:45:06.414480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:18:48.237612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337803+0000","last_change":"2024-03-16T20:45:05.424802+0000","last_active":"2024-03-16T20:45:07.337803+0000","last_peered":"2024-03-16T20:45:07.337803+0000","last_clean":"2024-03-16T20:45:07.337803+0000","last_became_active":"2024-03-16T20:45:05.424388+0000","last_became_peered":"2024-03-16T20:45:05.424388+0000","last_unstale":"2024-03-16T20:45:07.337803+0000","last_undegraded":"2024-03-16T20:45:07.337803+0000","last_fullsized":"2024-03-16T20:45:07.337803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:20:09.822409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.460115+0000","last_change":"2024-03-16T20:45:05.418605+0000","last_active":"2024-03-16T20:45:06.460115+0000","last_peered":"2024-03-16T20:45:06.460115+0000","last_clean":"2024-03-16T20:45:06.460115+0000","last_became_active":"2024-03-16T20:45:05.418293+0000","last_became_peered":"2024-03-16T20:45:05.418293+0000","last_unstale":"2024-03-16T20:45:06.460115+0000","last_undegraded":"2024-03-16T20:45:06.460115+0000","last_fullsized":"2024-03-16T20:45:06.460115+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:57:45.943288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412374+0000","last_change":"2024-03-16T20:45:05.422968+0000","last_active":"2024-03-16T20:45:06.412374+0000","last_peered":"2024-03-16T20:45:06.412374+0000","last_clean":"2024-03-16T20:45:06.412374+0000","last_became_active":"2024-03-16T20:45:05.422612+0000","last_became_peered":"2024-03-16T20:45:05.422612+0000","last_unstale":"2024-03-16T20:45:06.412374+0000","last_undegraded":"2024-03-16T20:45:06.412374+0000","last_fullsized":"2024-03-16T20:45:06.412374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:26:51.023909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337190+0000","last_change":"2024-03-16T20:45:05.426370+0000","last_active":"2024-03-16T20:45:07.337190+0000","last_peered":"2024-03-16T20:45:07.337190+0000","last_clean":"2024-03-16T20:45:07.337190+0000","last_became_active":"2024-03-16T20:45:05.426035+0000","last_became_peered":"2024-03-16T20:45:05.426035+0000","last_unstale":"2024-03-16T20:45:07.337190+0000","last_undegraded":"2024-03-16T20:45:07.337190+0000","last_fullsized":"2024-03-16T20:45:07.337190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:06:46.922690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413393+0000","last_change":"2024-03-16T20:45:05.410178+0000","last_active":"2024-03-16T20:45:06.413393+0000","last_peered":"2024-03-16T20:45:06.413393+0000","last_clean":"2024-03-16T20:45:06.413393+0000","last_became_active":"2024-03-16T20:45:05.409867+0000","last_became_peered":"2024-03-16T20:45:05.409867+0000","last_unstale":"2024-03-16T20:45:06.413393+0000","last_undegraded":"2024-03-16T20:45:06.413393+0000","last_fullsized":"2024-03-16T20:45:06.413393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:54.931097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.999686+0000","last_change":"2024-03-16T20:45:05.413326+0000","last_active":"2024-03-16T20:45:06.999686+0000","last_peered":"2024-03-16T20:45:06.999686+0000","last_clean":"2024-03-16T20:45:06.999686+0000","last_became_active":"2024-03-16T20:45:05.412843+0000","last_became_peered":"2024-03-16T20:45:05.412843+0000","last_unstale":"2024-03-16T20:45:06.999686+0000","last_undegraded":"2024-03-16T20:45:06.999686+0000","last_fullsized":"2024-03-16T20:45:06.999686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:03:04.229071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412143+0000","last_change":"2024-03-16T20:45:05.404852+0000","last_active":"2024-03-16T20:45:06.412143+0000","last_peered":"2024-03-16T20:45:06.412143+0000","last_clean":"2024-03-16T20:45:06.412143+0000","last_became_active":"2024-03-16T20:45:05.403728+0000","last_became_peered":"2024-03-16T20:45:05.403728+0000","last_unstale":"2024-03-16T20:45:06.412143+0000","last_undegraded":"2024-03-16T20:45:06.412143+0000","last_fullsized":"2024-03-16T20:45:06.412143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:29:10.023600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000158+0000","last_change":"2024-03-16T20:45:05.414436+0000","last_active":"2024-03-16T20:45:07.000158+0000","last_peered":"2024-03-16T20:45:07.000158+0000","last_clean":"2024-03-16T20:45:07.000158+0000","last_became_active":"2024-03-16T20:45:05.413903+0000","last_became_peered":"2024-03-16T20:45:05.413903+0000","last_unstale":"2024-03-16T20:45:07.000158+0000","last_undegraded":"2024-03-16T20:45:07.000158+0000","last_fullsized":"2024-03-16T20:45:07.000158+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:29:10.023600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413175+0000","last_change":"2024-03-16T20:45:05.423967+0000","last_active":"2024-03-16T20:45:06.413175+0000","last_peered":"2024-03-16T20:45:06.413175+0000","last_clean":"2024-03-16T20:45:06.413175+0000","last_became_active":"2024-03-16T20:45:05.423691+0000","last_became_peered":"2024-03-16T20:45:05.423691+0000","last_unstale":"2024-03-16T20:45:06.413175+0000","last_undegraded":"2024-03-16T20:45:06.413175+0000","last_fullsized":"2024-03-16T20:45:06.413175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:29:10.023600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410152+0000","last_change":"2024-03-16T20:45:05.418695+0000","last_active":"2024-03-16T20:45:06.410152+0000","last_peered":"2024-03-16T20:45:06.410152+0000","last_clean":"2024-03-16T20:45:06.410152+0000","last_became_active":"2024-03-16T20:45:05.415980+0000","last_became_peered":"2024-03-16T20:45:05.415980+0000","last_unstale":"2024-03-16T20:45:06.410152+0000","last_undegraded":"2024-03-16T20:45:06.410152+0000","last_fullsized":"2024-03-16T20:45:06.410152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:54:36.635249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000421+0000","last_change":"2024-03-16T20:45:05.414732+0000","last_active":"2024-03-16T20:45:07.000421+0000","last_peered":"2024-03-16T20:45:07.000421+0000","last_clean":"2024-03-16T20:45:07.000421+0000","last_became_active":"2024-03-16T20:45:05.411367+0000","last_became_peered":"2024-03-16T20:45:05.411367+0000","last_unstale":"2024-03-16T20:45:07.000421+0000","last_undegraded":"2024-03-16T20:45:07.000421+0000","last_fullsized":"2024-03-16T20:45:07.000421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:55:30.648219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337472+0000","last_change":"2024-03-16T20:45:05.422746+0000","last_active":"2024-03-16T20:45:07.337472+0000","last_peered":"2024-03-16T20:45:07.337472+0000","last_clean":"2024-03-16T20:45:07.337472+0000","last_became_active":"2024-03-16T20:45:05.422339+0000","last_became_peered":"2024-03-16T20:45:05.422339+0000","last_unstale":"2024-03-16T20:45:07.337472+0000","last_undegraded":"2024-03-16T20:45:07.337472+0000","last_fullsized":"2024-03-16T20:45:07.337472+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:15:27.440101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413606+0000","last_change":"2024-03-16T20:45:05.419051+0000","last_active":"2024-03-16T20:45:06.413606+0000","last_peered":"2024-03-16T20:45:06.413606+0000","last_clean":"2024-03-16T20:45:06.413606+0000","last_became_active":"2024-03-16T20:45:05.417745+0000","last_became_peered":"2024-03-16T20:45:05.417745+0000","last_unstale":"2024-03-16T20:45:06.413606+0000","last_undegraded":"2024-03-16T20:45:06.413606+0000","last_fullsized":"2024-03-16T20:45:06.413606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:15:41.403418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.340377+0000","last_change":"2024-03-16T20:45:05.409514+0000","last_active":"2024-03-16T20:45:07.340377+0000","last_peered":"2024-03-16T20:45:07.340377+0000","last_clean":"2024-03-16T20:45:07.340377+0000","last_became_active":"2024-03-16T20:45:05.408695+0000","last_became_peered":"2024-03-16T20:45:05.408695+0000","last_unstale":"2024-03-16T20:45:07.340377+0000","last_undegraded":"2024-03-16T20:45:07.340377+0000","last_fullsized":"2024-03-16T20:45:07.340377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:31:58.827839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.411788+0000","last_change":"2024-03-16T20:45:05.417983+0000","last_active":"2024-03-16T20:45:06.411788+0000","last_peered":"2024-03-16T20:45:06.411788+0000","last_clean":"2024-03-16T20:45:06.411788+0000","last_became_active":"2024-03-16T20:45:05.417669+0000","last_became_peered":"2024-03-16T20:45:05.417669+0000","last_unstale":"2024-03-16T20:45:06.411788+0000","last_undegraded":"2024-03-16T20:45:06.411788+0000","last_fullsized":"2024-03-16T20:45:06.411788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:15:27.440101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.336150+0000","last_change":"2024-03-16T20:45:05.407932+0000","last_active":"2024-03-16T20:45:07.336150+0000","last_peered":"2024-03-16T20:45:07.336150+0000","last_clean":"2024-03-16T20:45:07.336150+0000","last_became_active":"2024-03-16T20:45:05.407504+0000","last_became_peered":"2024-03-16T20:45:05.407504+0000","last_unstale":"2024-03-16T20:45:07.336150+0000","last_undegraded":"2024-03-16T20:45:07.336150+0000","last_fullsized":"2024-03-16T20:45:07.336150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:23:33.686287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000584+0000","last_change":"2024-03-16T20:45:04.380160+0000","last_active":"2024-03-16T20:45:07.000584+0000","last_peered":"2024-03-16T20:45:07.000584+0000","last_clean":"2024-03-16T20:45:07.000584+0000","last_became_active":"2024-03-16T20:45:04.379622+0000","last_became_peered":"2024-03-16T20:45:04.379622+0000","last_unstale":"2024-03-16T20:45:07.000584+0000","last_undegraded":"2024-03-16T20:45:07.000584+0000","last_fullsized":"2024-03-16T20:45:07.000584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:57.399645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337841+0000","last_change":"2024-03-16T20:45:05.421374+0000","last_active":"2024-03-16T20:45:07.337841+0000","last_peered":"2024-03-16T20:45:07.337841+0000","last_clean":"2024-03-16T20:45:07.337841+0000","last_became_active":"2024-03-16T20:45:05.420935+0000","last_became_peered":"2024-03-16T20:45:05.420935+0000","last_unstale":"2024-03-16T20:45:07.337841+0000","last_undegraded":"2024-03-16T20:45:07.337841+0000","last_fullsized":"2024-03-16T20:45:07.337841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:26:51.023909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412742+0000","last_change":"2024-03-16T20:45:04.389177+0000","last_active":"2024-03-16T20:45:06.412742+0000","last_peered":"2024-03-16T20:45:06.412742+0000","last_clean":"2024-03-16T20:45:06.412742+0000","last_became_active":"2024-03-16T20:45:04.388433+0000","last_became_peered":"2024-03-16T20:45:04.388433+0000","last_unstale":"2024-03-16T20:45:06.412742+0000","last_undegraded":"2024-03-16T20:45:06.412742+0000","last_fullsized":"2024-03-16T20:45:06.412742+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:48:48.175266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.999581+0000","last_change":"2024-03-16T20:45:04.383850+0000","last_active":"2024-03-16T20:45:06.999581+0000","last_peered":"2024-03-16T20:45:06.999581+0000","last_clean":"2024-03-16T20:45:06.999581+0000","last_became_active":"2024-03-16T20:45:04.383588+0000","last_became_peered":"2024-03-16T20:45:04.383588+0000","last_unstale":"2024-03-16T20:45:06.999581+0000","last_undegraded":"2024-03-16T20:45:06.999581+0000","last_fullsized":"2024-03-16T20:45:06.999581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:50:16.547893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.999693+0000","last_change":"2024-03-16T20:45:05.414605+0000","last_active":"2024-03-16T20:45:06.999693+0000","last_peered":"2024-03-16T20:45:06.999693+0000","last_clean":"2024-03-16T20:45:06.999693+0000","last_became_active":"2024-03-16T20:45:05.414067+0000","last_became_peered":"2024-03-16T20:45:05.414067+0000","last_unstale":"2024-03-16T20:45:06.999693+0000","last_undegraded":"2024-03-16T20:45:06.999693+0000","last_fullsized":"2024-03-16T20:45:06.999693+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:26:51.023909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412395+0000","last_change":"2024-03-16T20:45:04.387317+0000","last_active":"2024-03-16T20:45:06.412395+0000","last_peered":"2024-03-16T20:45:06.412395+0000","last_clean":"2024-03-16T20:45:06.412395+0000","last_became_active":"2024-03-16T20:45:04.386147+0000","last_became_peered":"2024-03-16T20:45:04.386147+0000","last_unstale":"2024-03-16T20:45:06.412395+0000","last_undegraded":"2024-03-16T20:45:06.412395+0000","last_fullsized":"2024-03-16T20:45:06.412395+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:48:48.175266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410049+0000","last_change":"2024-03-16T20:45:05.419053+0000","last_active":"2024-03-16T20:45:06.410049+0000","last_peered":"2024-03-16T20:45:06.410049+0000","last_clean":"2024-03-16T20:45:06.410049+0000","last_became_active":"2024-03-16T20:45:05.418610+0000","last_became_peered":"2024-03-16T20:45:05.418610+0000","last_unstale":"2024-03-16T20:45:06.410049+0000","last_undegraded":"2024-03-16T20:45:06.410049+0000","last_fullsized":"2024-03-16T20:45:06.410049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:03:04.229071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.414172+0000","last_change":"2024-03-16T20:45:04.400713+0000","last_active":"2024-03-16T20:45:06.414172+0000","last_peered":"2024-03-16T20:45:06.414172+0000","last_clean":"2024-03-16T20:45:06.414172+0000","last_became_active":"2024-03-16T20:45:04.399907+0000","last_became_peered":"2024-03-16T20:45:04.399907+0000","last_unstale":"2024-03-16T20:45:06.414172+0000","last_undegraded":"2024-03-16T20:45:06.414172+0000","last_fullsized":"2024-03-16T20:45:06.414172+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:34:16.171892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413409+0000","last_change":"2024-03-16T20:45:05.422773+0000","last_active":"2024-03-16T20:45:06.413409+0000","last_peered":"2024-03-16T20:45:06.413409+0000","last_clean":"2024-03-16T20:45:06.413409+0000","last_became_active":"2024-03-16T20:45:05.421962+0000","last_became_peered":"2024-03-16T20:45:05.421962+0000","last_unstale":"2024-03-16T20:45:06.413409+0000","last_undegraded":"2024-03-16T20:45:06.413409+0000","last_fullsized":"2024-03-16T20:45:06.413409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:11:19.673375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413369+0000","last_change":"2024-03-16T20:45:04.395629+0000","last_active":"2024-03-16T20:45:06.413369+0000","last_peered":"2024-03-16T20:45:06.413369+0000","last_clean":"2024-03-16T20:45:06.413369+0000","last_became_active":"2024-03-16T20:45:04.395107+0000","last_became_peered":"2024-03-16T20:45:04.395107+0000","last_unstale":"2024-03-16T20:45:06.413369+0000","last_undegraded":"2024-03-16T20:45:06.413369+0000","last_fullsized":"2024-03-16T20:45:06.413369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:03:03.237004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.339518+0000","last_change":"2024-03-16T20:45:05.421014+0000","last_active":"2024-03-16T20:45:07.339518+0000","last_peered":"2024-03-16T20:45:07.339518+0000","last_clean":"2024-03-16T20:45:07.339518+0000","last_became_active":"2024-03-16T20:45:05.420652+0000","last_became_peered":"2024-03-16T20:45:05.420652+0000","last_unstale":"2024-03-16T20:45:07.339518+0000","last_undegraded":"2024-03-16T20:45:07.339518+0000","last_fullsized":"2024-03-16T20:45:07.339518+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:35:07.902704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410232+0000","last_change":"2024-03-16T20:45:04.396880+0000","last_active":"2024-03-16T20:45:06.410232+0000","last_peered":"2024-03-16T20:45:06.410232+0000","last_clean":"2024-03-16T20:45:06.410232+0000","last_became_active":"2024-03-16T20:45:04.396654+0000","last_became_peered":"2024-03-16T20:45:04.396654+0000","last_unstale":"2024-03-16T20:45:06.410232+0000","last_undegraded":"2024-03-16T20:45:06.410232+0000","last_fullsized":"2024-03-16T20:45:06.410232+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:03:03.237004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.336336+0000","last_change":"2024-03-16T20:45:05.425535+0000","last_active":"2024-03-16T20:45:07.336336+0000","last_peered":"2024-03-16T20:45:07.336336+0000","last_clean":"2024-03-16T20:45:07.336336+0000","last_became_active":"2024-03-16T20:45:05.425236+0000","last_became_peered":"2024-03-16T20:45:05.425236+0000","last_unstale":"2024-03-16T20:45:07.336336+0000","last_undegraded":"2024-03-16T20:45:07.336336+0000","last_fullsized":"2024-03-16T20:45:07.336336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:48:49.183385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412682+0000","last_change":"2024-03-16T20:45:04.384514+0000","last_active":"2024-03-16T20:45:06.412682+0000","last_peered":"2024-03-16T20:45:06.412682+0000","last_clean":"2024-03-16T20:45:06.412682+0000","last_became_active":"2024-03-16T20:45:04.379541+0000","last_became_peered":"2024-03-16T20:45:04.379541+0000","last_unstale":"2024-03-16T20:45:06.412682+0000","last_undegraded":"2024-03-16T20:45:06.412682+0000","last_fullsized":"2024-03-16T20:45:06.412682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:28:48.672077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459401+0000","last_change":"2024-03-16T20:45:05.416547+0000","last_active":"2024-03-16T20:45:06.459401+0000","last_peered":"2024-03-16T20:45:06.459401+0000","last_clean":"2024-03-16T20:45:06.459401+0000","last_became_active":"2024-03-16T20:45:05.416216+0000","last_became_peered":"2024-03-16T20:45:05.416216+0000","last_unstale":"2024-03-16T20:45:06.459401+0000","last_undegraded":"2024-03-16T20:45:06.459401+0000","last_fullsized":"2024-03-16T20:45:06.459401+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:11:19.673375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.338023+0000","last_change":"2024-03-16T20:45:04.378488+0000","last_active":"2024-03-16T20:45:07.338023+0000","last_peered":"2024-03-16T20:45:07.338023+0000","last_clean":"2024-03-16T20:45:07.338023+0000","last_became_active":"2024-03-16T20:45:04.377819+0000","last_became_peered":"2024-03-16T20:45:04.377819+0000","last_unstale":"2024-03-16T20:45:07.338023+0000","last_undegraded":"2024-03-16T20:45:07.338023+0000","last_fullsized":"2024-03-16T20:45:07.338023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:09:39.701069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337814+0000","last_change":"2024-03-16T20:45:05.421867+0000","last_active":"2024-03-16T20:45:07.337814+0000","last_peered":"2024-03-16T20:45:07.337814+0000","last_clean":"2024-03-16T20:45:07.337814+0000","last_became_active":"2024-03-16T20:45:05.421377+0000","last_became_peered":"2024-03-16T20:45:05.421377+0000","last_unstale":"2024-03-16T20:45:07.337814+0000","last_undegraded":"2024-03-16T20:45:07.337814+0000","last_fullsized":"2024-03-16T20:45:07.337814+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:03:04.229071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413633+0000","last_change":"2024-03-16T20:45:04.401205+0000","last_active":"2024-03-16T20:45:06.413633+0000","last_peered":"2024-03-16T20:45:06.413633+0000","last_clean":"2024-03-16T20:45:06.413633+0000","last_became_active":"2024-03-16T20:45:04.396416+0000","last_became_peered":"2024-03-16T20:45:04.396416+0000","last_unstale":"2024-03-16T20:45:06.413633+0000","last_undegraded":"2024-03-16T20:45:06.413633+0000","last_fullsized":"2024-03-16T20:45:06.413633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:28:48.672077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459532+0000","last_change":"2024-03-16T20:45:05.421976+0000","last_active":"2024-03-16T20:45:06.459532+0000","last_peered":"2024-03-16T20:45:06.459532+0000","last_clean":"2024-03-16T20:45:06.459532+0000","last_became_active":"2024-03-16T20:45:05.421370+0000","last_became_peered":"2024-03-16T20:45:05.421370+0000","last_unstale":"2024-03-16T20:45:06.459532+0000","last_undegraded":"2024-03-16T20:45:06.459532+0000","last_fullsized":"2024-03-16T20:45:06.459532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:15:41.403418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.411884+0000","last_change":"2024-03-16T20:45:04.389332+0000","last_active":"2024-03-16T20:45:06.411884+0000","last_peered":"2024-03-16T20:45:06.411884+0000","last_clean":"2024-03-16T20:45:06.411884+0000","last_became_active":"2024-03-16T20:45:04.388661+0000","last_became_peered":"2024-03-16T20:45:04.388661+0000","last_unstale":"2024-03-16T20:45:06.411884+0000","last_undegraded":"2024-03-16T20:45:06.411884+0000","last_fullsized":"2024-03-16T20:45:06.411884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:23:32.678168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412210+0000","last_change":"2024-03-16T20:45:05.418621+0000","last_active":"2024-03-16T20:45:06.412210+0000","last_peered":"2024-03-16T20:45:06.412210+0000","last_clean":"2024-03-16T20:45:06.412210+0000","last_became_active":"2024-03-16T20:45:05.418292+0000","last_became_peered":"2024-03-16T20:45:05.418292+0000","last_unstale":"2024-03-16T20:45:06.412210+0000","last_undegraded":"2024-03-16T20:45:06.412210+0000","last_fullsized":"2024-03-16T20:45:06.412210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:22:42.961272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337037+0000","last_change":"2024-03-16T20:45:04.359171+0000","last_active":"2024-03-16T20:45:07.337037+0000","last_peered":"2024-03-16T20:45:07.337037+0000","last_clean":"2024-03-16T20:45:07.337037+0000","last_became_active":"2024-03-16T20:45:04.358863+0000","last_became_peered":"2024-03-16T20:45:04.358863+0000","last_unstale":"2024-03-16T20:45:07.337037+0000","last_undegraded":"2024-03-16T20:45:07.337037+0000","last_fullsized":"2024-03-16T20:45:07.337037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:28:48.672077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459047+0000","last_change":"2024-03-16T20:45:05.417061+0000","last_active":"2024-03-16T20:45:06.459047+0000","last_peered":"2024-03-16T20:45:06.459047+0000","last_clean":"2024-03-16T20:45:06.459047+0000","last_became_active":"2024-03-16T20:45:05.416454+0000","last_became_peered":"2024-03-16T20:45:05.416454+0000","last_unstale":"2024-03-16T20:45:06.459047+0000","last_undegraded":"2024-03-16T20:45:06.459047+0000","last_fullsized":"2024-03-16T20:45:06.459047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:55:30.648219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.336372+0000","last_change":"2024-03-16T20:45:04.376055+0000","last_active":"2024-03-16T20:45:07.336372+0000","last_peered":"2024-03-16T20:45:07.336372+0000","last_clean":"2024-03-16T20:45:07.336372+0000","last_became_active":"2024-03-16T20:45:04.375272+0000","last_became_peered":"2024-03-16T20:45:04.375272+0000","last_unstale":"2024-03-16T20:45:07.336372+0000","last_undegraded":"2024-03-16T20:45:07.336372+0000","last_fullsized":"2024-03-16T20:45:07.336372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:52:31.895539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459647+0000","last_change":"2024-03-16T20:45:05.420665+0000","last_active":"2024-03-16T20:45:06.459647+0000","last_peered":"2024-03-16T20:45:06.459647+0000","last_clean":"2024-03-16T20:45:06.459647+0000","last_became_active":"2024-03-16T20:45:05.420363+0000","last_became_peered":"2024-03-16T20:45:05.420363+0000","last_unstale":"2024-03-16T20:45:06.459647+0000","last_undegraded":"2024-03-16T20:45:06.459647+0000","last_fullsized":"2024-03-16T20:45:06.459647+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:59:02.640185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337985+0000","last_change":"2024-03-16T20:45:04.363877+0000","last_active":"2024-03-16T20:45:07.337985+0000","last_peered":"2024-03-16T20:45:07.337985+0000","last_clean":"2024-03-16T20:45:07.337985+0000","last_became_active":"2024-03-16T20:45:04.363180+0000","last_became_peered":"2024-03-16T20:45:04.363180+0000","last_unstale":"2024-03-16T20:45:07.337985+0000","last_undegraded":"2024-03-16T20:45:07.337985+0000","last_fullsized":"2024-03-16T20:45:07.337985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:57.399645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.460378+0000","last_change":"2024-03-16T20:45:05.429416+0000","last_active":"2024-03-16T20:45:06.460378+0000","last_peered":"2024-03-16T20:45:06.460378+0000","last_clean":"2024-03-16T20:45:06.460378+0000","last_became_active":"2024-03-16T20:45:05.428766+0000","last_became_peered":"2024-03-16T20:45:05.428766+0000","last_unstale":"2024-03-16T20:45:06.460378+0000","last_undegraded":"2024-03-16T20:45:06.460378+0000","last_fullsized":"2024-03-16T20:45:06.460378+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:45:30.989735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410331+0000","last_change":"2024-03-16T20:45:04.355156+0000","last_active":"2024-03-16T20:45:06.410331+0000","last_peered":"2024-03-16T20:45:06.410331+0000","last_clean":"2024-03-16T20:45:06.410331+0000","last_became_active":"2024-03-16T20:45:04.354621+0000","last_became_peered":"2024-03-16T20:45:04.354621+0000","last_unstale":"2024-03-16T20:45:06.410331+0000","last_undegraded":"2024-03-16T20:45:06.410331+0000","last_fullsized":"2024-03-16T20:45:06.410331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:31:57.819721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413605+0000","last_change":"2024-03-16T20:45:05.419655+0000","last_active":"2024-03-16T20:45:06.413605+0000","last_peered":"2024-03-16T20:45:06.413605+0000","last_clean":"2024-03-16T20:45:06.413605+0000","last_became_active":"2024-03-16T20:45:05.417669+0000","last_became_peered":"2024-03-16T20:45:05.417669+0000","last_unstale":"2024-03-16T20:45:06.413605+0000","last_undegraded":"2024-03-16T20:45:06.413605+0000","last_fullsized":"2024-03-16T20:45:06.413605+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:59:02.640185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.338415+0000","last_change":"2024-03-16T20:45:04.401781+0000","last_active":"2024-03-16T20:45:07.338415+0000","last_peered":"2024-03-16T20:45:07.338415+0000","last_clean":"2024-03-16T20:45:07.338415+0000","last_became_active":"2024-03-16T20:45:04.401563+0000","last_became_peered":"2024-03-16T20:45:04.401563+0000","last_unstale":"2024-03-16T20:45:07.338415+0000","last_undegraded":"2024-03-16T20:45:07.338415+0000","last_fullsized":"2024-03-16T20:45:07.338415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:34:16.171892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459736+0000","last_change":"2024-03-16T20:45:05.413949+0000","last_active":"2024-03-16T20:45:06.459736+0000","last_peered":"2024-03-16T20:45:06.459736+0000","last_clean":"2024-03-16T20:45:06.459736+0000","last_became_active":"2024-03-16T20:45:05.413609+0000","last_became_peered":"2024-03-16T20:45:05.413609+0000","last_unstale":"2024-03-16T20:45:06.459736+0000","last_undegraded":"2024-03-16T20:45:06.459736+0000","last_fullsized":"2024-03-16T20:45:06.459736+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:18:48.237612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":170,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.458981+0000","last_change":"2024-03-16T20:44:54.293418+0000","last_active":"2024-03-16T20:45:06.458981+0000","last_peered":"2024-03-16T20:45:06.458981+0000","last_clean":"2024-03-16T20:45:06.458981+0000","last_became_active":"2024-03-16T20:44:54.291347+0000","last_became_peered":"2024-03-16T20:44:54.291347+0000","last_unstale":"2024-03-16T20:45:06.458981+0000","last_undegraded":"2024-03-16T20:45:06.458981+0000","last_fullsized":"2024-03-16T20:45:06.458981+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_clean_scrub_stamp":"2024-03-16T20:44:53.077639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:29:18.318494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412202+0000","last_change":"2024-03-16T20:45:04.394662+0000","last_active":"2024-03-16T20:45:06.412202+0000","last_peered":"2024-03-16T20:45:06.412202+0000","last_clean":"2024-03-16T20:45:06.412202+0000","last_became_active":"2024-03-16T20:45:04.394119+0000","last_became_peered":"2024-03-16T20:45:04.394119+0000","last_unstale":"2024-03-16T20:45:06.412202+0000","last_undegraded":"2024-03-16T20:45:06.412202+0000","last_fullsized":"2024-03-16T20:45:06.412202+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:34:16.171892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000045+0000","last_change":"2024-03-16T20:45:05.413913+0000","last_active":"2024-03-16T20:45:07.000045+0000","last_peered":"2024-03-16T20:45:07.000045+0000","last_clean":"2024-03-16T20:45:07.000045+0000","last_became_active":"2024-03-16T20:45:05.412780+0000","last_became_peered":"2024-03-16T20:45:05.412780+0000","last_unstale":"2024-03-16T20:45:07.000045+0000","last_undegraded":"2024-03-16T20:45:07.000045+0000","last_fullsized":"2024-03-16T20:45:07.000045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:54:36.635249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.458865+0000","last_change":"2024-03-16T20:45:04.380237+0000","last_active":"2024-03-16T20:45:06.458865+0000","last_peered":"2024-03-16T20:45:06.458865+0000","last_clean":"2024-03-16T20:45:06.458865+0000","last_became_active":"2024-03-16T20:45:04.379086+0000","last_became_peered":"2024-03-16T20:45:04.379086+0000","last_unstale":"2024-03-16T20:45:06.458865+0000","last_undegraded":"2024-03-16T20:45:06.458865+0000","last_fullsized":"2024-03-16T20:45:06.458865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:23:32.678168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.336799+0000","last_change":"2024-03-16T20:45:05.414361+0000","last_active":"2024-03-16T20:45:07.336799+0000","last_peered":"2024-03-16T20:45:07.336799+0000","last_clean":"2024-03-16T20:45:07.336799+0000","last_became_active":"2024-03-16T20:45:05.411900+0000","last_became_peered":"2024-03-16T20:45:05.411900+0000","last_unstale":"2024-03-16T20:45:07.336799+0000","last_undegraded":"2024-03-16T20:45:07.336799+0000","last_fullsized":"2024-03-16T20:45:07.336799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:16:59.281226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.340530+0000","last_change":"2024-03-16T20:45:04.363289+0000","last_active":"2024-03-16T20:45:07.340530+0000","last_peered":"2024-03-16T20:45:07.340530+0000","last_clean":"2024-03-16T20:45:07.340530+0000","last_became_active":"2024-03-16T20:45:04.362919+0000","last_became_peered":"2024-03-16T20:45:04.362919+0000","last_unstale":"2024-03-16T20:45:07.340530+0000","last_undegraded":"2024-03-16T20:45:07.340530+0000","last_fullsized":"2024-03-16T20:45:07.340530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:50:16.547893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.409757+0000","last_change":"2024-03-16T20:45:05.408984+0000","last_active":"2024-03-16T20:45:06.409757+0000","last_peered":"2024-03-16T20:45:06.409757+0000","last_clean":"2024-03-16T20:45:06.409757+0000","last_became_active":"2024-03-16T20:45:05.408648+0000","last_became_peered":"2024-03-16T20:45:05.408648+0000","last_unstale":"2024-03-16T20:45:06.409757+0000","last_undegraded":"2024-03-16T20:45:06.409757+0000","last_fullsized":"2024-03-16T20:45:06.409757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:22:42.961272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413903+0000","last_change":"2024-03-16T20:45:04.394583+0000","last_active":"2024-03-16T20:45:06.413903+0000","last_peered":"2024-03-16T20:45:06.413903+0000","last_clean":"2024-03-16T20:45:06.413903+0000","last_became_active":"2024-03-16T20:45:04.394016+0000","last_became_peered":"2024-03-16T20:45:04.394016+0000","last_unstale":"2024-03-16T20:45:06.413903+0000","last_undegraded":"2024-03-16T20:45:06.413903+0000","last_fullsized":"2024-03-16T20:45:06.413903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:23:32.678168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.336052+0000","last_change":"2024-03-16T20:45:05.425183+0000","last_active":"2024-03-16T20:45:07.336052+0000","last_peered":"2024-03-16T20:45:07.336052+0000","last_clean":"2024-03-16T20:45:07.336052+0000","last_became_active":"2024-03-16T20:45:05.424862+0000","last_became_peered":"2024-03-16T20:45:05.424862+0000","last_unstale":"2024-03-16T20:45:07.336052+0000","last_undegraded":"2024-03-16T20:45:07.336052+0000","last_fullsized":"2024-03-16T20:45:07.336052+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:03:04.245122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410481+0000","last_change":"2024-03-16T20:45:04.354012+0000","last_active":"2024-03-16T20:45:06.410481+0000","last_peered":"2024-03-16T20:45:06.410481+0000","last_clean":"2024-03-16T20:45:06.410481+0000","last_became_active":"2024-03-16T20:45:04.353589+0000","last_became_peered":"2024-03-16T20:45:04.353589+0000","last_unstale":"2024-03-16T20:45:06.410481+0000","last_undegraded":"2024-03-16T20:45:06.410481+0000","last_fullsized":"2024-03-16T20:45:06.410481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:48:48.175266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.415027+0000","last_change":"2024-03-16T20:45:05.417774+0000","last_active":"2024-03-16T20:45:06.415027+0000","last_peered":"2024-03-16T20:45:06.415027+0000","last_clean":"2024-03-16T20:45:06.415027+0000","last_became_active":"2024-03-16T20:45:05.417404+0000","last_became_peered":"2024-03-16T20:45:05.417404+0000","last_unstale":"2024-03-16T20:45:06.415027+0000","last_undegraded":"2024-03-16T20:45:06.415027+0000","last_fullsized":"2024-03-16T20:45:06.415027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:57:45.943288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459976+0000","last_change":"2024-03-16T20:45:04.381406+0000","last_active":"2024-03-16T20:45:06.459976+0000","last_peered":"2024-03-16T20:45:06.459976+0000","last_clean":"2024-03-16T20:45:06.459976+0000","last_became_active":"2024-03-16T20:45:04.380608+0000","last_became_peered":"2024-03-16T20:45:04.380608+0000","last_unstale":"2024-03-16T20:45:06.459976+0000","last_undegraded":"2024-03-16T20:45:06.459976+0000","last_fullsized":"2024-03-16T20:45:06.459976+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:16:39.193904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410306+0000","last_change":"2024-03-16T20:45:05.408949+0000","last_active":"2024-03-16T20:45:06.410306+0000","last_peered":"2024-03-16T20:45:06.410306+0000","last_clean":"2024-03-16T20:45:06.410306+0000","last_became_active":"2024-03-16T20:45:05.408619+0000","last_became_peered":"2024-03-16T20:45:05.408619+0000","last_unstale":"2024-03-16T20:45:06.410306+0000","last_undegraded":"2024-03-16T20:45:06.410306+0000","last_fullsized":"2024-03-16T20:45:06.410306+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:35:07.902704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.611129+0000","last_change":"2024-03-16T20:45:04.383772+0000","last_active":"2024-03-16T20:45:06.611129+0000","last_peered":"2024-03-16T20:45:06.611129+0000","last_clean":"2024-03-16T20:45:06.611129+0000","last_became_active":"2024-03-16T20:45:04.383050+0000","last_became_peered":"2024-03-16T20:45:04.383050+0000","last_unstale":"2024-03-16T20:45:06.611129+0000","last_undegraded":"2024-03-16T20:45:06.611129+0000","last_fullsized":"2024-03-16T20:45:06.611129+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:20:08.814290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000319+0000","last_change":"2024-03-16T20:45:05.419648+0000","last_active":"2024-03-16T20:45:07.000319+0000","last_peered":"2024-03-16T20:45:07.000319+0000","last_clean":"2024-03-16T20:45:07.000319+0000","last_became_active":"2024-03-16T20:45:05.419358+0000","last_became_peered":"2024-03-16T20:45:05.419358+0000","last_unstale":"2024-03-16T20:45:07.000319+0000","last_undegraded":"2024-03-16T20:45:07.000319+0000","last_fullsized":"2024-03-16T20:45:07.000319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:50:22.920748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.414177+0000","last_change":"2024-03-16T20:45:04.382826+0000","last_active":"2024-03-16T20:45:06.414177+0000","last_peered":"2024-03-16T20:45:06.414177+0000","last_clean":"2024-03-16T20:45:06.414177+0000","last_became_active":"2024-03-16T20:45:04.381824+0000","last_became_peered":"2024-03-16T20:45:04.381824+0000","last_unstale":"2024-03-16T20:45:06.414177+0000","last_undegraded":"2024-03-16T20:45:06.414177+0000","last_fullsized":"2024-03-16T20:45:06.414177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:03:03.237004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.337732+0000","last_change":"2024-03-16T20:45:05.415644+0000","last_active":"2024-03-16T20:45:07.337732+0000","last_peered":"2024-03-16T20:45:07.337732+0000","last_clean":"2024-03-16T20:45:07.337732+0000","last_became_active":"2024-03-16T20:45:05.415294+0000","last_became_peered":"2024-03-16T20:45:05.415294+0000","last_unstale":"2024-03-16T20:45:07.337732+0000","last_undegraded":"2024-03-16T20:45:07.337732+0000","last_fullsized":"2024-03-16T20:45:07.337732+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:22:42.961272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.458835+0000","last_change":"2024-03-16T20:45:04.380383+0000","last_active":"2024-03-16T20:45:06.458835+0000","last_peered":"2024-03-16T20:45:06.458835+0000","last_clean":"2024-03-16T20:45:06.458835+0000","last_became_active":"2024-03-16T20:45:04.379121+0000","last_became_peered":"2024-03-16T20:45:04.379121+0000","last_unstale":"2024-03-16T20:45:06.458835+0000","last_undegraded":"2024-03-16T20:45:06.458835+0000","last_fullsized":"2024-03-16T20:45:06.458835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:03:03.237004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.336388+0000","last_change":"2024-03-16T20:45:05.408820+0000","last_active":"2024-03-16T20:45:07.336388+0000","last_peered":"2024-03-16T20:45:07.336388+0000","last_clean":"2024-03-16T20:45:07.336388+0000","last_became_active":"2024-03-16T20:45:05.408337+0000","last_became_peered":"2024-03-16T20:45:05.408337+0000","last_unstale":"2024-03-16T20:45:07.336388+0000","last_undegraded":"2024-03-16T20:45:07.336388+0000","last_fullsized":"2024-03-16T20:45:07.336388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:31:58.827839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410303+0000","last_change":"2024-03-16T20:45:04.379953+0000","last_active":"2024-03-16T20:45:06.410303+0000","last_peered":"2024-03-16T20:45:06.410303+0000","last_clean":"2024-03-16T20:45:06.410303+0000","last_became_active":"2024-03-16T20:45:04.379610+0000","last_became_peered":"2024-03-16T20:45:04.379610+0000","last_unstale":"2024-03-16T20:45:06.410303+0000","last_undegraded":"2024-03-16T20:45:06.410303+0000","last_fullsized":"2024-03-16T20:45:06.410303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:23:32.678168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000614+0000","last_change":"2024-03-16T20:45:05.401685+0000","last_active":"2024-03-16T20:45:07.000614+0000","last_peered":"2024-03-16T20:45:07.000614+0000","last_clean":"2024-03-16T20:45:07.000614+0000","last_became_active":"2024-03-16T20:45:05.399878+0000","last_became_peered":"2024-03-16T20:45:05.399878+0000","last_unstale":"2024-03-16T20:45:07.000614+0000","last_undegraded":"2024-03-16T20:45:07.000614+0000","last_fullsized":"2024-03-16T20:45:07.000614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:11:19.673375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.410121+0000","last_change":"2024-03-16T20:45:04.359852+0000","last_active":"2024-03-16T20:45:06.410121+0000","last_peered":"2024-03-16T20:45:06.410121+0000","last_clean":"2024-03-16T20:45:06.410121+0000","last_became_active":"2024-03-16T20:45:04.359557+0000","last_became_peered":"2024-03-16T20:45:04.359557+0000","last_unstale":"2024-03-16T20:45:06.410121+0000","last_undegraded":"2024-03-16T20:45:06.410121+0000","last_fullsized":"2024-03-16T20:45:06.410121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:34:16.171892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000547+0000","last_change":"2024-03-16T20:45:05.422880+0000","last_active":"2024-03-16T20:45:07.000547+0000","last_peered":"2024-03-16T20:45:07.000547+0000","last_clean":"2024-03-16T20:45:07.000547+0000","last_became_active":"2024-03-16T20:45:05.422585+0000","last_became_peered":"2024-03-16T20:45:05.422585+0000","last_unstale":"2024-03-16T20:45:07.000547+0000","last_undegraded":"2024-03-16T20:45:07.000547+0000","last_fullsized":"2024-03-16T20:45:07.000547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:30:54.931097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459415+0000","last_change":"2024-03-16T20:45:04.382087+0000","last_active":"2024-03-16T20:45:06.459415+0000","last_peered":"2024-03-16T20:45:06.459415+0000","last_clean":"2024-03-16T20:45:06.459415+0000","last_became_active":"2024-03-16T20:45:04.381331+0000","last_became_peered":"2024-03-16T20:45:04.381331+0000","last_unstale":"2024-03-16T20:45:06.459415+0000","last_undegraded":"2024-03-16T20:45:06.459415+0000","last_fullsized":"2024-03-16T20:45:06.459415+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:31:57.819721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.412636+0000","last_change":"2024-03-16T20:45:05.416889+0000","last_active":"2024-03-16T20:45:06.412636+0000","last_peered":"2024-03-16T20:45:06.412636+0000","last_clean":"2024-03-16T20:45:06.412636+0000","last_became_active":"2024-03-16T20:45:05.416560+0000","last_became_peered":"2024-03-16T20:45:05.416560+0000","last_unstale":"2024-03-16T20:45:06.412636+0000","last_undegraded":"2024-03-16T20:45:06.412636+0000","last_fullsized":"2024-03-16T20:45:06.412636+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:54:36.635249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.414635+0000","last_change":"2024-03-16T20:45:05.403182+0000","last_active":"2024-03-16T20:45:06.414635+0000","last_peered":"2024-03-16T20:45:06.414635+0000","last_clean":"2024-03-16T20:45:06.414635+0000","last_became_active":"2024-03-16T20:45:05.401656+0000","last_became_peered":"2024-03-16T20:45:05.401656+0000","last_unstale":"2024-03-16T20:45:06.414635+0000","last_undegraded":"2024-03-16T20:45:06.414635+0000","last_fullsized":"2024-03-16T20:45:06.414635+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:40:17.773165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459476+0000","last_change":"2024-03-16T20:45:04.402308+0000","last_active":"2024-03-16T20:45:06.459476+0000","last_peered":"2024-03-16T20:45:06.459476+0000","last_clean":"2024-03-16T20:45:06.459476+0000","last_became_active":"2024-03-16T20:45:04.401750+0000","last_became_peered":"2024-03-16T20:45:04.401750+0000","last_unstale":"2024-03-16T20:45:06.459476+0000","last_undegraded":"2024-03-16T20:45:06.459476+0000","last_fullsized":"2024-03-16T20:45:06.459476+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:48:48.175266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.413317+0000","last_change":"2024-03-16T20:45:05.412664+0000","last_active":"2024-03-16T20:45:06.413317+0000","last_peered":"2024-03-16T20:45:06.413317+0000","last_clean":"2024-03-16T20:45:06.413317+0000","last_became_active":"2024-03-16T20:45:05.412302+0000","last_became_peered":"2024-03-16T20:45:05.412302+0000","last_unstale":"2024-03-16T20:45:06.413317+0000","last_undegraded":"2024-03-16T20:45:06.413317+0000","last_fullsized":"2024-03-16T20:45:06.413317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:55:30.648219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:06.459690+0000","last_change":"2024-03-16T20:45:04.387378+0000","last_active":"2024-03-16T20:45:06.459690+0000","last_peered":"2024-03-16T20:45:06.459690+0000","last_clean":"2024-03-16T20:45:06.459690+0000","last_became_active":"2024-03-16T20:45:04.386602+0000","last_became_peered":"2024-03-16T20:45:04.386602+0000","last_unstale":"2024-03-16T20:45:06.459690+0000","last_undegraded":"2024-03-16T20:45:06.459690+0000","last_fullsized":"2024-03-16T20:45:06.459690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:20:08.814290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.339116+0000","last_change":"2024-03-16T20:45:05.413222+0000","last_active":"2024-03-16T20:45:07.339116+0000","last_peered":"2024-03-16T20:45:07.339116+0000","last_clean":"2024-03-16T20:45:07.339116+0000","last_became_active":"2024-03-16T20:45:05.412620+0000","last_became_peered":"2024-03-16T20:45:05.412620+0000","last_unstale":"2024-03-16T20:45:07.339116+0000","last_undegraded":"2024-03-16T20:45:07.339116+0000","last_fullsized":"2024-03-16T20:45:07.339116+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:54:36.635249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000425+0000","last_change":"2024-03-16T20:45:04.362033+0000","last_active":"2024-03-16T20:45:07.000425+0000","last_peered":"2024-03-16T20:45:07.000425+0000","last_clean":"2024-03-16T20:45:07.000425+0000","last_became_active":"2024-03-16T20:45:04.361100+0000","last_became_peered":"2024-03-16T20:45:04.361100+0000","last_unstale":"2024-03-16T20:45:07.000425+0000","last_undegraded":"2024-03-16T20:45:07.000425+0000","last_fullsized":"2024-03-16T20:45:07.000425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:28:48.672077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.000396+0000","last_change":"2024-03-16T20:45:05.425640+0000","last_active":"2024-03-16T20:45:07.000396+0000","last_peered":"2024-03-16T20:45:07.000396+0000","last_clean":"2024-03-16T20:45:07.000396+0000","last_became_active":"2024-03-16T20:45:05.424729+0000","last_became_peered":"2024-03-16T20:45:05.424729+0000","last_unstale":"2024-03-16T20:45:07.000396+0000","last_undegraded":"2024-03-16T20:45:07.000396+0000","last_fullsized":"2024-03-16T20:45:07.000396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:04.339420+0000","last_clean_scrub_stamp":"2024-03-16T20:45:04.339420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:26:54.432519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-03-16T20:45:07.339770+0000","last_change":"2024-03-16T20:45:04.380444+0000","last_active":"2024-03-16T20:45:07.339770+0000","last_peered":"2024-03-16T20:45:07.339770+0000","last_clean":"2024-03-16T20:45:07.339770+0000","last_became_active":"2024-03-16T20:45:04.380164+0000","last_became_peered":"2024-03-16T20:45:04.380164+0000","last_unstale":"2024-03-16T20:45:07.339770+0000","last_undegraded":"2024-03-16T20:45:07.339770+0000","last_fullsized":"2024-03-16T20:45:07.339770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:03.331302+0000","last_clean_scrub_stamp":"2024-03-16T20:45:03.331302+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:43:46.634405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31980,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339860,"statfs":{"total":96636764160,"available":96604016640,"internally_reserved":0,"allocated":815104,"data_stored":666555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31412,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340428,"statfs":{"total":96636764160,"available":96604598272,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":75731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":77217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":75605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":75765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27828,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344012,"statfs":{"total":96636764160,"available":96608268288,"internally_reserved":0,"allocated":823296,"data_stored":666611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T20:45:12.910 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-16T20:45:12.910 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-16T20:45:12.910 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-03-16T20:45:12.910 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-03-16T20:45:13.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.056+0000 7faf51090640 1 Processor -- start 2024-03-16T20:45:13.059 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.056+0000 7faf51090640 1 -- start start 2024-03-16T20:45:13.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.056+0000 7faf51090640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7faf4c13ed60 0x7faf4c13f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.056+0000 7faf51090640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7faf4c148430 0x7faf4c148880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.056+0000 7faf51090640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4c0b02a0 con 0x7faf4c072120 2024-03-16T20:45:13.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.056+0000 7faf51090640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4c148dc0 con 0x7faf4c13ed60 2024-03-16T20:45:13.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.056+0000 7faf51090640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4c0b1050 con 0x7faf4c148430 2024-03-16T20:45:13.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.056+0000 7faf4a59c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7faf4c13ed60 0x7faf4c13f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.056+0000 7faf4a59c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7faf4c13ed60 0x7faf4c13f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41650/0 (socket says 172.21.15.122:41650) 2024-03-16T20:45:13.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.056+0000 7faf4a59c640 1 -- 172.21.15.122:0/1788553038 learned_addr learned my addr 172.21.15.122:0/1788553038 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:13.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.056+0000 7faf4b59e640 1 --2- 172.21.15.122:0/1788553038 >> v2:172.21.15.195:3300/0 conn(0x7faf4c148430 0x7faf4c148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.056+0000 7faf4a59c640 1 -- 172.21.15.122:0/1788553038 >> v2:172.21.15.195:3300/0 conn(0x7faf4c148430 msgr2=0x7faf4c148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:13.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.056+0000 7faf4a59c640 1 --2- 172.21.15.122:0/1788553038 >> v2:172.21.15.195:3300/0 conn(0x7faf4c148430 0x7faf4c148880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.061 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.056+0000 7faf4a59c640 1 -- 172.21.15.122:0/1788553038 >> v1:172.21.15.195:6789/0 conn(0x7faf4c072120 legacy=0x7faf4c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:13.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.056+0000 7faf4a59c640 1 -- 172.21.15.122:0/1788553038 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c072500 con 0x7faf4c13ed60 2024-03-16T20:45:13.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.060+0000 7faf4a59c640 1 --2- 172.21.15.122:0/1788553038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf4c13ed60 0x7faf4c13f1c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faf4c148f00 tx=0x7faf3c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5492d5c095d4a87e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:13.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.060+0000 7faf49d9b640 1 -- 172.21.15.122:0/1788553038 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c0ab600 con 0x7faf4c13ed60 2024-03-16T20:45:13.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.060+0000 7faf49d9b640 1 -- 172.21.15.122:0/1788553038 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3c0a7050 con 0x7faf4c13ed60 2024-03-16T20:45:13.063 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.060+0000 7faf49d9b640 1 -- 172.21.15.122:0/1788553038 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c0ab980 con 0x7faf4c13ed60 2024-03-16T20:45:13.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.060+0000 7faf51090640 1 -- 172.21.15.122:0/1788553038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf4c13ed60 msgr2=0x7faf4c13f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.060+0000 7faf51090640 1 --2- 172.21.15.122:0/1788553038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf4c13ed60 0x7faf4c13f1c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faf4c148f00 tx=0x7faf3c01c450 comp rx=0 tx=0).stop 2024-03-16T20:45:13.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.060+0000 7faf51090640 1 -- 172.21.15.122:0/1788553038 shutdown_connections 2024-03-16T20:45:13.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.060+0000 7faf51090640 1 --2- 172.21.15.122:0/1788553038 >> v2:172.21.15.195:3300/0 conn(0x7faf4c148430 0x7faf4c148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.060+0000 7faf51090640 1 --2- 172.21.15.122:0/1788553038 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf4c13ed60 0x7faf4c13f1c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.060+0000 7faf51090640 1 -- 172.21.15.122:0/1788553038 >> 172.21.15.122:0/1788553038 conn(0x7faf4c0b8860 msgr2=0x7faf4c0b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:13.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.060+0000 7faf51090640 1 -- 172.21.15.122:0/1788553038 shutdown_connections 2024-03-16T20:45:13.064 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.060+0000 7faf51090640 1 -- 172.21.15.122:0/1788553038 wait complete. 2024-03-16T20:45:13.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.060+0000 7faf51090640 1 Processor -- start 2024-03-16T20:45:13.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.060+0000 7faf51090640 1 -- start start 2024-03-16T20:45:13.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf51090640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf4c072120 0x7faf4c147a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.067 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf51090640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf4c13ed60 0x7faf4c1dd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf51090640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf4c148430 0x7faf4c1ddd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf51090640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c1e1a60 con 0x7faf4c148430 2024-03-16T20:45:13.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf51090640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c1de290 con 0x7faf4c072120 2024-03-16T20:45:13.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf51090640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c14eb90 con 0x7faf4c13ed60 2024-03-16T20:45:13.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf4b59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf4c148430 0x7faf4c1ddd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf4b59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf4c148430 0x7faf4c1ddd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41660/0 (socket says 172.21.15.122:41660) 2024-03-16T20:45:13.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf4b59e640 1 -- 172.21.15.122:0/4214267005 learned_addr learned my addr 172.21.15.122:0/4214267005 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:13.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf4a59c640 1 --2- 172.21.15.122:0/4214267005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf4c13ed60 0x7faf4c1dd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf4ad9d640 1 --2- 172.21.15.122:0/4214267005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf4c072120 0x7faf4c147a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf4b59e640 1 -- 172.21.15.122:0/4214267005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf4c13ed60 msgr2=0x7faf4c1dd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf4b59e640 1 --2- 172.21.15.122:0/4214267005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf4c13ed60 0x7faf4c1dd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf4b59e640 1 -- 172.21.15.122:0/4214267005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf4c072120 msgr2=0x7faf4c147a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf4b59e640 1 --2- 172.21.15.122:0/4214267005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf4c072120 0x7faf4c147a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf4b59e640 1 -- 172.21.15.122:0/4214267005 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c0094d0 con 0x7faf4c148430 2024-03-16T20:45:13.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf4ad9d640 1 --2- 172.21.15.122:0/4214267005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf4c072120 0x7faf4c147a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:13.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf4b59e640 1 --2- 172.21.15.122:0/4214267005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf4c148430 0x7faf4c1ddd50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faf4400d560 tx=0x7faf4400da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:13.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf437fe640 1 -- 172.21.15.122:0/4214267005 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf440125f0 con 0x7faf4c148430 2024-03-16T20:45:13.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf437fe640 1 -- 172.21.15.122:0/4214267005 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf44012770 con 0x7faf4c148430 2024-03-16T20:45:13.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf437fe640 1 -- 172.21.15.122:0/4214267005 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44012b20 con 0x7faf4c148430 2024-03-16T20:45:13.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf51090640 1 -- 172.21.15.122:0/4214267005 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c0af7e0 con 0x7faf4c148430 2024-03-16T20:45:13.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf51090640 1 -- 172.21.15.122:0/4214267005 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c13fa10 con 0x7faf4c148430 2024-03-16T20:45:13.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf51090640 1 -- 172.21.15.122:0/4214267005 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4c1e1ba0 con 0x7faf4c148430 2024-03-16T20:45:13.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf437fe640 1 -- 172.21.15.122:0/4214267005 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7faf44012c80 con 0x7faf4c148430 2024-03-16T20:45:13.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf437fe640 1 --2- 172.21.15.122:0/4214267005 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7faf2c066e40 0x7faf2c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf437fe640 1 -- 172.21.15.122:0/4214267005 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf44084e50 con 0x7faf4c148430 2024-03-16T20:45:13.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf4ad9d640 1 --2- 172.21.15.122:0/4214267005 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7faf2c066e40 0x7faf2c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.064+0000 7faf4ad9d640 1 --2- 172.21.15.122:0/4214267005 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7faf2c066e40 0x7faf2c069300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faf34007fd0 tx=0x7faf34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:13.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.068+0000 7faf437fe640 1 -- 172.21.15.122:0/4214267005 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7faf4c1e1ba0 con 0x7faf4c148430 2024-03-16T20:45:13.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.248+0000 7faf51090640 1 -- 172.21.15.122:0/4214267005 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7faf4c1e1ba0 con 0x7faf4c148430 2024-03-16T20:45:13.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.248+0000 7faf437fe640 1 -- 172.21.15.122:0/4214267005 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7faf4c1e1ba0 con 0x7faf4c148430 2024-03-16T20:45:13.251 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:13.251 INFO:teuthology.orchestra.run.smithi122.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-16T20:45:13.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.248+0000 7faf51090640 1 -- 172.21.15.122:0/4214267005 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7faf2c066e40 msgr2=0x7faf2c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.248+0000 7faf51090640 1 --2- 172.21.15.122:0/4214267005 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7faf2c066e40 0x7faf2c069300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7faf34007fd0 tx=0x7faf34009040 comp rx=0 tx=0).stop 2024-03-16T20:45:13.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.248+0000 7faf51090640 1 -- 172.21.15.122:0/4214267005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf4c148430 msgr2=0x7faf4c1ddd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.248+0000 7faf51090640 1 --2- 172.21.15.122:0/4214267005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf4c148430 0x7faf4c1ddd50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faf4400d560 tx=0x7faf4400da20 comp rx=0 tx=0).stop 2024-03-16T20:45:13.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.248+0000 7faf51090640 1 -- 172.21.15.122:0/4214267005 shutdown_connections 2024-03-16T20:45:13.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.248+0000 7faf51090640 1 --2- 172.21.15.122:0/4214267005 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7faf2c066e40 0x7faf2c069300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.248+0000 7faf51090640 1 --2- 172.21.15.122:0/4214267005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faf4c148430 0x7faf4c1ddd50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.248+0000 7faf51090640 1 --2- 172.21.15.122:0/4214267005 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7faf4c13ed60 0x7faf4c1dd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.248+0000 7faf51090640 1 --2- 172.21.15.122:0/4214267005 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7faf4c072120 0x7faf4c147a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.248+0000 7faf51090640 1 -- 172.21.15.122:0/4214267005 >> 172.21.15.122:0/4214267005 conn(0x7faf4c0b8860 msgr2=0x7faf4c142e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:13.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.248+0000 7faf51090640 1 -- 172.21.15.122:0/4214267005 shutdown_connections 2024-03-16T20:45:13.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.248+0000 7faf51090640 1 -- 172.21.15.122:0/4214267005 wait complete. 2024-03-16T20:45:13.268 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-03-16T20:45:13.268 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.380+0000 7f50c33b2640 1 Processor -- start 2024-03-16T20:45:13.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.380+0000 7f50c33b2640 1 -- start start 2024-03-16T20:45:13.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.380+0000 7f50c33b2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50bc072120 0x7f50bc0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.380+0000 7f50c33b2640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50bc0b1050 con 0x7f50bc148b40 2024-03-16T20:45:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.380+0000 7f50c33b2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50bc072500 con 0x7f50bc13ed60 2024-03-16T20:45:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.380+0000 7f50c33b2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc0b38a0 con 0x7f50bc072120 2024-03-16T20:45:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.380+0000 7f50c194f640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f50bc148b40 0x7f50bc14f780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58662/0 (socket says 172.21.15.122:58662) 2024-03-16T20:45:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.380+0000 7f50c194f640 1 -- 172.21.15.122:0/1110552455 learned_addr learned my addr 172.21.15.122:0/1110552455 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.380+0000 7f50c114e640 1 --2- 172.21.15.122:0/1110552455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50bc072120 0x7f50bc0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 -- 172.21.15.122:0/1110552455 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853479738 0 0) 0x7f50bc0b1050 con 0x7f50bc148b40 2024-03-16T20:45:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 -- 172.21.15.122:0/1110552455 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50a8003530 con 0x7f50bc148b40 2024-03-16T20:45:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 -- 172.21.15.122:0/1110552455 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2298559034 0 0) 0x7f50a8003530 con 0x7f50bc148b40 2024-03-16T20:45:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 -- 172.21.15.122:0/1110552455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50bc072120 msgr2=0x7f50bc0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 --2- 172.21.15.122:0/1110552455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50bc072120 0x7f50bc0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 -- 172.21.15.122:0/1110552455 >> v1:172.21.15.195:6789/0 conn(0x7f50bc13ed60 legacy=0x7f50bc148430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 -- 172.21.15.122:0/1110552455 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc0b3b80 con 0x7f50bc148b40 2024-03-16T20:45:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 -- 172.21.15.122:0/1110552455 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f50b4001a00 con 0x7f50bc148b40 2024-03-16T20:45:13.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 -- 172.21.15.122:0/1110552455 >> v1:172.21.15.122:6789/0 conn(0x7f50bc148b40 legacy=0x7f50bc14f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 --2- 172.21.15.122:0/1110552455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50a80039b0 0x7f50a8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 --2- 172.21.15.122:0/1110552455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a80047d0 0x7f50a8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 --2- 172.21.15.122:0/1110552455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50bc072120 0x7f50a80051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 -- 172.21.15.122:0/1110552455 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc0b38a0 con 0x7f50a80047d0 2024-03-16T20:45:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 -- 172.21.15.122:0/1110552455 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a800bac0 con 0x7f50a80039b0 2024-03-16T20:45:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 -- 172.21.15.122:0/1110552455 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a800bc80 con 0x7f50bc072120 2024-03-16T20:45:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50c094d640 1 --2- 172.21.15.122:0/1110552455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a80047d0 0x7f50a8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50c114e640 1 --2- 172.21.15.122:0/1110552455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50a80039b0 0x7f50a8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50c094d640 1 -- 172.21.15.122:0/1110552455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50bc072120 msgr2=0x7f50a80051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50c094d640 1 --2- 172.21.15.122:0/1110552455 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50bc072120 0x7f50a80051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50c094d640 1 -- 172.21.15.122:0/1110552455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50a80039b0 msgr2=0x7f50a8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50c094d640 1 --2- 172.21.15.122:0/1110552455 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50a80039b0 0x7f50a8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50c094d640 1 -- 172.21.15.122:0/1110552455 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a8003530 con 0x7f50a80047d0 2024-03-16T20:45:13.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50c094d640 1 --2- 172.21.15.122:0/1110552455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a80047d0 0x7f50a8004c50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f50ac001780 tx=0x7f50ac01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21af8980901b267f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:13.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 -- 172.21.15.122:0/1110552455 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac0052d0 con 0x7f50a80047d0 2024-03-16T20:45:13.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 -- 172.21.15.122:0/1110552455 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50ac0a7050 con 0x7f50a80047d0 2024-03-16T20:45:13.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50b3fff640 1 -- 172.21.15.122:0/1110552455 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac003600 con 0x7f50a80047d0 2024-03-16T20:45:13.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50c33b2640 1 -- 172.21.15.122:0/1110552455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a80047d0 msgr2=0x7f50a8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50c33b2640 1 --2- 172.21.15.122:0/1110552455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a80047d0 0x7f50a8004c50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f50ac001780 tx=0x7f50ac01c3f0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50c194f640 1 -- 172.21.15.122:0/1110552455 reap_dead start 2024-03-16T20:45:13.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50c33b2640 1 -- 172.21.15.122:0/1110552455 shutdown_connections 2024-03-16T20:45:13.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50c33b2640 1 -- 172.21.15.122:0/1110552455 >> 172.21.15.122:0/1110552455 conn(0x7f50bc0b8860 msgr2=0x7f50bc0b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:13.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50c33b2640 1 -- 172.21.15.122:0/1110552455 shutdown_connections 2024-03-16T20:45:13.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.384+0000 7f50c33b2640 1 -- 172.21.15.122:0/1110552455 wait complete. 2024-03-16T20:45:13.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c33b2640 1 Processor -- start 2024-03-16T20:45:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c33b2640 1 -- start start 2024-03-16T20:45:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c33b2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a80047d0 0x7f50bc0b7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c33b2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50bc13e140 0x7f50bc13cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c33b2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50bc13e5a0 0x7f50bc1335d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c33b2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc0af8a0 con 0x7f50a80047d0 2024-03-16T20:45:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c33b2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc137080 con 0x7f50bc13e5a0 2024-03-16T20:45:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c33b2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc0af5e0 con 0x7f50bc13e140 2024-03-16T20:45:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c114e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a80047d0 0x7f50bc0b7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c114e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a80047d0 0x7f50bc0b7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41670/0 (socket says 172.21.15.122:41670) 2024-03-16T20:45:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c114e640 1 -- 172.21.15.122:0/2235805803 learned_addr learned my addr 172.21.15.122:0/2235805803 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c094d640 1 --2- 172.21.15.122:0/2235805803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50bc13e140 0x7f50bc13cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c194f640 1 --2- 172.21.15.122:0/2235805803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50bc13e5a0 0x7f50bc1335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c114e640 1 -- 172.21.15.122:0/2235805803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50bc13e140 msgr2=0x7f50bc13cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c114e640 1 --2- 172.21.15.122:0/2235805803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50bc13e140 0x7f50bc13cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c114e640 1 -- 172.21.15.122:0/2235805803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50bc13e5a0 msgr2=0x7f50bc1335d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c114e640 1 --2- 172.21.15.122:0/2235805803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50bc13e5a0 0x7f50bc1335d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c114e640 1 -- 172.21.15.122:0/2235805803 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50bc13d580 con 0x7f50a80047d0 2024-03-16T20:45:13.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c094d640 1 --2- 172.21.15.122:0/2235805803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50bc13e140 0x7f50bc13cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:13.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c114e640 1 --2- 172.21.15.122:0/2235805803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a80047d0 0x7f50bc0b7f60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f50b80115e0 tx=0x7f50b8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:13.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50b1ffb640 1 -- 172.21.15.122:0/2235805803 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b800f8f0 con 0x7f50a80047d0 2024-03-16T20:45:13.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50b1ffb640 1 -- 172.21.15.122:0/2235805803 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b800fa50 con 0x7f50a80047d0 2024-03-16T20:45:13.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c33b2640 1 -- 172.21.15.122:0/2235805803 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50bc13c5c0 con 0x7f50a80047d0 2024-03-16T20:45:13.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50b1ffb640 1 -- 172.21.15.122:0/2235805803 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b800fdc0 con 0x7f50a80047d0 2024-03-16T20:45:13.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c33b2640 1 -- 172.21.15.122:0/2235805803 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50bc13d020 con 0x7f50a80047d0 2024-03-16T20:45:13.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50c33b2640 1 -- 172.21.15.122:0/2235805803 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50bc075280 con 0x7f50a80047d0 2024-03-16T20:45:13.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50b1ffb640 1 -- 172.21.15.122:0/2235805803 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f50b8015460 con 0x7f50a80047d0 2024-03-16T20:45:13.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50b1ffb640 1 --2- 172.21.15.122:0/2235805803 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f50a0066e40 0x7f50a0069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.388+0000 7f50b1ffb640 1 -- 172.21.15.122:0/2235805803 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50b80852c0 con 0x7f50a80047d0 2024-03-16T20:45:13.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.392+0000 7f50c094d640 1 --2- 172.21.15.122:0/2235805803 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f50a0066e40 0x7f50a0069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.392+0000 7f50c094d640 1 --2- 172.21.15.122:0/2235805803 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f50a0066e40 0x7f50a0069300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f50ac001750 tx=0x7f50ac0011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:13.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.392+0000 7f50b1ffb640 1 -- 172.21.15.122:0/2235805803 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f50b8052140 con 0x7f50a80047d0 2024-03-16T20:45:13.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.544+0000 7f50c33b2640 1 -- 172.21.15.122:0/2235805803 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50bc13dd60 con 0x7f50a80047d0 2024-03-16T20:45:13.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.548+0000 7f50b1ffb640 1 -- 172.21.15.122:0/2235805803 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f50b8055800 con 0x7f50a80047d0 2024-03-16T20:45:13.552 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:13.552 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:06.404479+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.548+0000 7f50c33b2640 1 -- 172.21.15.122:0/2235805803 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f50a0066e40 msgr2=0x7f50a0069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.548+0000 7f50c33b2640 1 --2- 172.21.15.122:0/2235805803 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f50a0066e40 0x7f50a0069300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f50ac001750 tx=0x7f50ac0011a0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.548+0000 7f50c33b2640 1 -- 172.21.15.122:0/2235805803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a80047d0 msgr2=0x7f50bc0b7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.552+0000 7f50c33b2640 1 --2- 172.21.15.122:0/2235805803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a80047d0 0x7f50bc0b7f60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f50b80115e0 tx=0x7f50b8011aa0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.552+0000 7f50c33b2640 1 -- 172.21.15.122:0/2235805803 shutdown_connections 2024-03-16T20:45:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.552+0000 7f50c33b2640 1 --2- 172.21.15.122:0/2235805803 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f50a0066e40 0x7f50a0069300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.552+0000 7f50c33b2640 1 --2- 172.21.15.122:0/2235805803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50bc13e5a0 0x7f50bc1335d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.552+0000 7f50c33b2640 1 --2- 172.21.15.122:0/2235805803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50bc13e140 0x7f50bc13cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.552+0000 7f50c33b2640 1 --2- 172.21.15.122:0/2235805803 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50a80047d0 0x7f50bc0b7f60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.552+0000 7f50c33b2640 1 -- 172.21.15.122:0/2235805803 >> 172.21.15.122:0/2235805803 conn(0x7f50bc0b8860 msgr2=0x7f50bc0b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:13.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.552+0000 7f50c33b2640 1 -- 172.21.15.122:0/2235805803 shutdown_connections 2024-03-16T20:45:13.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.552+0000 7f50c33b2640 1 -- 172.21.15.122:0/2235805803 wait complete. 2024-03-16T20:45:13.569 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T20:44:52.653523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T20:45:02.758515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T20:45:03.710783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T20:45:13.569 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T20:45:13.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa779949640 1 Processor -- start 2024-03-16T20:45:13.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa779949640 1 -- start start 2024-03-16T20:45:13.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa779949640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa77409ae30 0x7fa7740af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa779949640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7740b4020 0x7fa7740b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa779949640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa7740b7820 0x7fa7740b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa779949640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7740b38e0 con 0x7fa77409ae30 2024-03-16T20:45:13.679 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa779949640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77409b210 con 0x7fa7740b7820 2024-03-16T20:45:13.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa779949640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740b4fb0 con 0x7fa7740b4020 2024-03-16T20:45:13.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa772ffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa77409ae30 0x7fa7740af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa772ffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa77409ae30 0x7fa7740af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41676/0 (socket says 172.21.15.122:41676) 2024-03-16T20:45:13.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa772ffd640 1 -- 172.21.15.122:0/3921438054 learned_addr learned my addr 172.21.15.122:0/3921438054 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:13.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa7737fe640 1 --2- 172.21.15.122:0/3921438054 >> v2:172.21.15.195:3300/0 conn(0x7fa7740b7820 0x7fa7740b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa7727fc640 1 --2- 172.21.15.122:0/3921438054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7740b4020 0x7fa7740b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa772ffd640 1 -- 172.21.15.122:0/3921438054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7740b4020 msgr2=0x7fa7740b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:13.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa772ffd640 1 --2- 172.21.15.122:0/3921438054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7740b4020 0x7fa7740b67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa772ffd640 1 -- 172.21.15.122:0/3921438054 >> v2:172.21.15.195:3300/0 conn(0x7fa7740b7820 msgr2=0x7fa7740b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:13.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa772ffd640 1 --2- 172.21.15.122:0/3921438054 >> v2:172.21.15.195:3300/0 conn(0x7fa7740b7820 0x7fa7740b6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa772ffd640 1 -- 172.21.15.122:0/3921438054 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7740b3290 con 0x7fa77409ae30 2024-03-16T20:45:13.680 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa7737fe640 1 --2- 172.21.15.122:0/3921438054 >> v2:172.21.15.195:3300/0 conn(0x7fa7740b7820 0x7fa7740b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:13.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa772ffd640 1 --2- 172.21.15.122:0/3921438054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa77409ae30 0x7fa7740af080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa77406fb90 tx=0x7fa76801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97055b1ae52d8d42 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:13.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa771ffb640 1 -- 172.21.15.122:0/3921438054 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7680ab670 con 0x7fa77409ae30 2024-03-16T20:45:13.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa771ffb640 1 -- 172.21.15.122:0/3921438054 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7680ab7d0 con 0x7fa77409ae30 2024-03-16T20:45:13.681 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.676+0000 7fa771ffb640 1 -- 172.21.15.122:0/3921438054 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7680abb60 con 0x7fa77409ae30 2024-03-16T20:45:13.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 -- 172.21.15.122:0/3921438054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa77409ae30 msgr2=0x7fa7740af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 --2- 172.21.15.122:0/3921438054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa77409ae30 0x7fa7740af080 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa77406fb90 tx=0x7fa76801c490 comp rx=0 tx=0).stop 2024-03-16T20:45:13.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 -- 172.21.15.122:0/3921438054 shutdown_connections 2024-03-16T20:45:13.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 --2- 172.21.15.122:0/3921438054 >> v2:172.21.15.195:3300/0 conn(0x7fa7740b7820 0x7fa7740b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.682 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 --2- 172.21.15.122:0/3921438054 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa7740b4020 0x7fa7740b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 --2- 172.21.15.122:0/3921438054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa77409ae30 0x7fa7740af080 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 -- 172.21.15.122:0/3921438054 >> 172.21.15.122:0/3921438054 conn(0x7fa7740b8d10 msgr2=0x7fa77409a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:13.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 -- 172.21.15.122:0/3921438054 shutdown_connections 2024-03-16T20:45:13.683 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 -- 172.21.15.122:0/3921438054 wait complete. 2024-03-16T20:45:13.685 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 Processor -- start 2024-03-16T20:45:13.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 -- start start 2024-03-16T20:45:13.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa77409ae30 0x7fa774099db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7740b4020 0x7fa7740701f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7740b7820 0x7fa774071320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa774141e60 con 0x7fa7740b4020 2024-03-16T20:45:13.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa774072660 con 0x7fa7740b7820 2024-03-16T20:45:13.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa774071dc0 con 0x7fa77409ae30 2024-03-16T20:45:13.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa7727fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7740b4020 0x7fa7740701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa7727fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7740b4020 0x7fa7740701f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41678/0 (socket says 172.21.15.122:41678) 2024-03-16T20:45:13.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa7727fc640 1 -- 172.21.15.122:0/3612195484 learned_addr learned my addr 172.21.15.122:0/3612195484 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:13.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa772ffd640 1 --2- 172.21.15.122:0/3612195484 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa77409ae30 0x7fa774099db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa7737fe640 1 --2- 172.21.15.122:0/3612195484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7740b7820 0x7fa774071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa7727fc640 1 -- 172.21.15.122:0/3612195484 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa77409ae30 msgr2=0x7fa774099db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa7727fc640 1 --2- 172.21.15.122:0/3612195484 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa77409ae30 0x7fa774099db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa7727fc640 1 -- 172.21.15.122:0/3612195484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7740b7820 msgr2=0x7fa774071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa7727fc640 1 --2- 172.21.15.122:0/3612195484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7740b7820 0x7fa774071320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa7727fc640 1 -- 172.21.15.122:0/3612195484 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7680094d0 con 0x7fa7740b4020 2024-03-16T20:45:13.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa7737fe640 1 --2- 172.21.15.122:0/3612195484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7740b7820 0x7fa774071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:13.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa7727fc640 1 --2- 172.21.15.122:0/3612195484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7740b4020 0x7fa7740701f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c7f0 tx=0x7fa76400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:13.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa75bfff640 1 -- 172.21.15.122:0/3612195484 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764004220 con 0x7fa7740b4020 2024-03-16T20:45:13.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa75bfff640 1 -- 172.21.15.122:0/3612195484 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa764004380 con 0x7fa7740b4020 2024-03-16T20:45:13.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 -- 172.21.15.122:0/3612195484 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7740720a0 con 0x7fa7740b4020 2024-03-16T20:45:13.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa75bfff640 1 -- 172.21.15.122:0/3612195484 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764003840 con 0x7fa7740b4020 2024-03-16T20:45:13.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 -- 172.21.15.122:0/3612195484 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa774070730 con 0x7fa7740b4020 2024-03-16T20:45:13.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.680+0000 7fa779949640 1 -- 172.21.15.122:0/3612195484 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa774074bb0 con 0x7fa7740b4020 2024-03-16T20:45:13.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.684+0000 7fa75bfff640 1 -- 172.21.15.122:0/3612195484 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fa764003bb0 con 0x7fa7740b4020 2024-03-16T20:45:13.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.684+0000 7fa75bfff640 1 --2- 172.21.15.122:0/3612195484 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa760066e40 0x7fa760069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.684+0000 7fa75bfff640 1 -- 172.21.15.122:0/3612195484 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa764084970 con 0x7fa7740b4020 2024-03-16T20:45:13.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.684+0000 7fa772ffd640 1 --2- 172.21.15.122:0/3612195484 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa760066e40 0x7fa760069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.688 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.684+0000 7fa772ffd640 1 --2- 172.21.15.122:0/3612195484 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa760066e40 0x7fa760069300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa7680af000 tx=0x7fa7680a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:13.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.688+0000 7fa75bfff640 1 -- 172.21.15.122:0/3612195484 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa7640517f0 con 0x7fa7740b4020 2024-03-16T20:45:13.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.832+0000 7fa779949640 1 -- 172.21.15.122:0/3612195484 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa7740b49f0 con 0x7fa7740b4020 2024-03-16T20:45:13.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.832+0000 7fa75bfff640 1 -- 172.21.15.122:0/3612195484 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa76401e020 con 0x7fa7740b4020 2024-03-16T20:45:13.836 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-03-16T20:45:13.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.836+0000 7fa779949640 1 -- 172.21.15.122:0/3612195484 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa760066e40 msgr2=0x7fa760069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.836+0000 7fa779949640 1 --2- 172.21.15.122:0/3612195484 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa760066e40 0x7fa760069300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa7680af000 tx=0x7fa7680a7000 comp rx=0 tx=0).stop 2024-03-16T20:45:13.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.836+0000 7fa779949640 1 -- 172.21.15.122:0/3612195484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7740b4020 msgr2=0x7fa7740701f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.836+0000 7fa779949640 1 --2- 172.21.15.122:0/3612195484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7740b4020 0x7fa7740701f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa76400c7f0 tx=0x7fa76400ccb0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.836+0000 7fa779949640 1 -- 172.21.15.122:0/3612195484 shutdown_connections 2024-03-16T20:45:13.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.836+0000 7fa779949640 1 --2- 172.21.15.122:0/3612195484 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa760066e40 0x7fa760069300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.836+0000 7fa779949640 1 --2- 172.21.15.122:0/3612195484 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa7740b7820 0x7fa774071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.836+0000 7fa779949640 1 --2- 172.21.15.122:0/3612195484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa7740b4020 0x7fa7740701f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.836+0000 7fa779949640 1 --2- 172.21.15.122:0/3612195484 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa77409ae30 0x7fa774099db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.836+0000 7fa779949640 1 -- 172.21.15.122:0/3612195484 >> 172.21.15.122:0/3612195484 conn(0x7fa7740b8d10 msgr2=0x7fa7741451b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:13.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.836+0000 7fa779949640 1 -- 172.21.15.122:0/3612195484 shutdown_connections 2024-03-16T20:45:13.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.836+0000 7fa779949640 1 -- 172.21.15.122:0/3612195484 wait complete. 2024-03-16T20:45:13.853 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T20:45:13.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc807e97640 1 Processor -- start 2024-03-16T20:45:13.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc807e97640 1 -- start start 2024-03-16T20:45:13.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc807e97640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc8000b6510 0x7fc800142270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc807e97640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8000b3cc0 con 0x7fc80009ae30 2024-03-16T20:45:13.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc807e97640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8000b3a50 con 0x7fc8000b8240 2024-03-16T20:45:13.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc807e97640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc80009b210 con 0x7fc8000b6510 2024-03-16T20:45:13.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc805c33640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fc80009ae30 0x7fc8000b7b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58670/0 (socket says 172.21.15.122:58670) 2024-03-16T20:45:13.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc805c33640 1 -- 172.21.15.122:0/2266834585 learned_addr learned my addr 172.21.15.122:0/2266834585 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:13.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc805432640 1 --2- 172.21.15.122:0/2266834585 >> v2:172.21.15.195:3300/0 conn(0x7fc8000b6510 0x7fc800142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc804c31640 1 -- 172.21.15.122:0/2266834585 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1167256882 0 0) 0x7fc8000b3cc0 con 0x7fc80009ae30 2024-03-16T20:45:13.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc804c31640 1 -- 172.21.15.122:0/2266834585 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7ec003530 con 0x7fc80009ae30 2024-03-16T20:45:13.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc804c31640 1 -- 172.21.15.122:0/2266834585 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1630840662 0 0) 0x7fc7ec003530 con 0x7fc80009ae30 2024-03-16T20:45:13.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc804c31640 1 -- 172.21.15.122:0/2266834585 >> v2:172.21.15.195:3300/0 conn(0x7fc8000b6510 msgr2=0x7fc800142270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:13.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc804c31640 1 --2- 172.21.15.122:0/2266834585 >> v2:172.21.15.195:3300/0 conn(0x7fc8000b6510 0x7fc800142270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc804c31640 1 -- 172.21.15.122:0/2266834585 >> v1:172.21.15.195:6789/0 conn(0x7fc8000b8240 legacy=0x7fc80014f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc804c31640 1 -- 172.21.15.122:0/2266834585 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8000b49e0 con 0x7fc80009ae30 2024-03-16T20:45:13.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc804c31640 1 -- 172.21.15.122:0/2266834585 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7fc7fc0016a0 con 0x7fc80009ae30 2024-03-16T20:45:13.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc805432640 1 --2- 172.21.15.122:0/2266834585 >> v2:172.21.15.195:3300/0 conn(0x7fc8000b6510 0x7fc800142270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:13.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc804c31640 1 -- 172.21.15.122:0/2266834585 >> v1:172.21.15.122:6789/0 conn(0x7fc80009ae30 legacy=0x7fc8000b7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc804c31640 1 --2- 172.21.15.122:0/2266834585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7ec003980 0x7fc7ec003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc804c31640 1 --2- 172.21.15.122:0/2266834585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ec0047d0 0x7fc7ec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc804c31640 1 --2- 172.21.15.122:0/2266834585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ec0052b0 0x7fc7ec00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc804c31640 1 -- 172.21.15.122:0/2266834585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80009b210 con 0x7fc7ec0052b0 2024-03-16T20:45:13.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc804c31640 1 -- 172.21.15.122:0/2266834585 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ec00c180 con 0x7fc7ec0047d0 2024-03-16T20:45:13.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc804c31640 1 -- 172.21.15.122:0/2266834585 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7ec00c340 con 0x7fc7ec003980 2024-03-16T20:45:13.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc806434640 1 --2- 172.21.15.122:0/2266834585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ec0052b0 0x7fc7ec00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc805c33640 1 --2- 172.21.15.122:0/2266834585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7ec003980 0x7fc7ec003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc805432640 1 --2- 172.21.15.122:0/2266834585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ec0047d0 0x7fc7ec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc806434640 1 -- 172.21.15.122:0/2266834585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7ec003980 msgr2=0x7fc7ec003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:13.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc806434640 1 --2- 172.21.15.122:0/2266834585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7ec003980 0x7fc7ec003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc806434640 1 -- 172.21.15.122:0/2266834585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ec0047d0 msgr2=0x7fc7ec004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:13.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc806434640 1 --2- 172.21.15.122:0/2266834585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ec0047d0 0x7fc7ec004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.968+0000 7fc806434640 1 -- 172.21.15.122:0/2266834585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec003530 con 0x7fc7ec0052b0 2024-03-16T20:45:13.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc806434640 1 -- 172.21.15.122:0/2266834585 reap_dead start 2024-03-16T20:45:13.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc806434640 1 --2- 172.21.15.122:0/2266834585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ec0052b0 0x7fc7ec00b990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8001e40 tx=0x7fc7f801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1aa41f0daf99fa server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:13.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc804c31640 1 -- 172.21.15.122:0/2266834585 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8003b80 con 0x7fc7ec0052b0 2024-03-16T20:45:13.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc804c31640 1 -- 172.21.15.122:0/2266834585 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f8003ce0 con 0x7fc7ec0052b0 2024-03-16T20:45:13.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc804c31640 1 -- 172.21.15.122:0/2266834585 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f80052f0 con 0x7fc7ec0052b0 2024-03-16T20:45:13.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc807e97640 1 -- 172.21.15.122:0/2266834585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ec0052b0 msgr2=0x7fc7ec00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc807e97640 1 --2- 172.21.15.122:0/2266834585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ec0052b0 0x7fc7ec00b990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8001e40 tx=0x7fc7f801c760 comp rx=0 tx=0).stop 2024-03-16T20:45:13.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc807e97640 1 -- 172.21.15.122:0/2266834585 shutdown_connections 2024-03-16T20:45:13.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc807e97640 1 --2- 172.21.15.122:0/2266834585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7ec0052b0 0x7fc7ec00b990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc807e97640 1 -- 172.21.15.122:0/2266834585 >> 172.21.15.122:0/2266834585 conn(0x7fc8000b8d10 msgr2=0x7fc80009a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:13.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc807e97640 1 -- 172.21.15.122:0/2266834585 shutdown_connections 2024-03-16T20:45:13.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc807e97640 1 -- 172.21.15.122:0/2266834585 wait complete. 2024-03-16T20:45:13.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc807e97640 1 Processor -- start 2024-03-16T20:45:13.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc807e97640 1 -- start start 2024-03-16T20:45:13.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc807e97640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ec0052b0 0x7fc8000af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc807e97640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8000b05c0 0x7fc800098c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc807e97640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc800099160 0x7fc80013ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc807e97640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80009a260 con 0x7fc800099160 2024-03-16T20:45:13.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc807e97640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8000b2810 con 0x7fc7ec0052b0 2024-03-16T20:45:13.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc807e97640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8000afe30 con 0x7fc8000b05c0 2024-03-16T20:45:13.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc806434640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc800099160 0x7fc80013ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc806434640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc800099160 0x7fc80013ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41696/0 (socket says 172.21.15.122:41696) 2024-03-16T20:45:13.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc806434640 1 -- 172.21.15.122:0/967567643 learned_addr learned my addr 172.21.15.122:0/967567643 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:13.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc805c33640 1 --2- 172.21.15.122:0/967567643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ec0052b0 0x7fc8000af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc805432640 1 --2- 172.21.15.122:0/967567643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8000b05c0 0x7fc800098c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc806434640 1 -- 172.21.15.122:0/967567643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8000b05c0 msgr2=0x7fc800098c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc806434640 1 --2- 172.21.15.122:0/967567643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8000b05c0 0x7fc800098c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc806434640 1 -- 172.21.15.122:0/967567643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ec0052b0 msgr2=0x7fc8000af1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:13.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc806434640 1 --2- 172.21.15.122:0/967567643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ec0052b0 0x7fc8000af1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:13.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc806434640 1 -- 172.21.15.122:0/967567643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8000995e0 con 0x7fc800099160 2024-03-16T20:45:13.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc806434640 1 --2- 172.21.15.122:0/967567643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc800099160 0x7fc80013ee60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc7f801c7a0 tx=0x7fc7ec005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:13.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc7f67fc640 1 -- 172.21.15.122:0/967567643 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f80ae070 con 0x7fc800099160 2024-03-16T20:45:13.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc7f67fc640 1 -- 172.21.15.122:0/967567643 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f8001260 con 0x7fc800099160 2024-03-16T20:45:13.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc807e97640 1 -- 172.21.15.122:0/967567643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8000b0df0 con 0x7fc800099160 2024-03-16T20:45:13.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc7f67fc640 1 -- 172.21.15.122:0/967567643 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8006b60 con 0x7fc800099160 2024-03-16T20:45:13.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.972+0000 7fc807e97640 1 -- 172.21.15.122:0/967567643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8000afac0 con 0x7fc800099160 2024-03-16T20:45:13.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.976+0000 7fc807e97640 1 -- 172.21.15.122:0/967567643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc800074bb0 con 0x7fc800099160 2024-03-16T20:45:13.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.976+0000 7fc7f67fc640 1 -- 172.21.15.122:0/967567643 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fc7f80b3470 con 0x7fc800099160 2024-03-16T20:45:13.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.976+0000 7fc7f67fc640 1 --2- 172.21.15.122:0/967567643 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc7e4066e40 0x7fc7e4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:13.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.976+0000 7fc7f67fc640 1 -- 172.21.15.122:0/967567643 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7f811df90 con 0x7fc800099160 2024-03-16T20:45:13.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.976+0000 7fc805c33640 1 --2- 172.21.15.122:0/967567643 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc7e4066e40 0x7fc7e4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:13.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.976+0000 7fc805c33640 1 --2- 172.21.15.122:0/967567643 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc7e4066e40 0x7fc7e4069300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc000c10 tx=0x7fc7fc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:13.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:13.980+0000 7fc7f67fc640 1 -- 172.21.15.122:0/967567643 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc7f80eae10 con 0x7fc800099160 2024-03-16T20:45:14.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.124+0000 7fc807e97640 1 -- 172.21.15.122:0/967567643 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc8000aff70 con 0x7fc800099160 2024-03-16T20:45:14.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.124+0000 7fc7f67fc640 1 -- 172.21.15.122:0/967567643 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc7f80ee4d0 con 0x7fc800099160 2024-03-16T20:45:14.129 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-03-16T20:45:14.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.128+0000 7fc807e97640 1 -- 172.21.15.122:0/967567643 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc7e4066e40 msgr2=0x7fc7e4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.128+0000 7fc807e97640 1 --2- 172.21.15.122:0/967567643 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc7e4066e40 0x7fc7e4069300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc000c10 tx=0x7fc7fc009040 comp rx=0 tx=0).stop 2024-03-16T20:45:14.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.128+0000 7fc807e97640 1 -- 172.21.15.122:0/967567643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc800099160 msgr2=0x7fc80013ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.128+0000 7fc807e97640 1 --2- 172.21.15.122:0/967567643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc800099160 0x7fc80013ee60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc7f801c7a0 tx=0x7fc7ec005800 comp rx=0 tx=0).stop 2024-03-16T20:45:14.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.128+0000 7fc807e97640 1 -- 172.21.15.122:0/967567643 shutdown_connections 2024-03-16T20:45:14.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.128+0000 7fc807e97640 1 --2- 172.21.15.122:0/967567643 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc7e4066e40 0x7fc7e4069300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.128+0000 7fc807e97640 1 --2- 172.21.15.122:0/967567643 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc800099160 0x7fc80013ee60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.128+0000 7fc807e97640 1 --2- 172.21.15.122:0/967567643 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc8000b05c0 0x7fc800098c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.128+0000 7fc807e97640 1 --2- 172.21.15.122:0/967567643 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7ec0052b0 0x7fc8000af1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.128+0000 7fc807e97640 1 -- 172.21.15.122:0/967567643 >> 172.21.15.122:0/967567643 conn(0x7fc8000b8d10 msgr2=0x7fc80006d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:14.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.128+0000 7fc807e97640 1 -- 172.21.15.122:0/967567643 shutdown_connections 2024-03-16T20:45:14.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.128+0000 7fc807e97640 1 -- 172.21.15.122:0/967567643 wait complete. 2024-03-16T20:45:14.146 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T20:45:14.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.308+0000 7fd8ba301640 1 Processor -- start 2024-03-16T20:45:14.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.308+0000 7fd8ba301640 1 -- start start 2024-03-16T20:45:14.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8ba301640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd8b409ae30 0x7fd8b40b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8ba301640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd8b41429f0 0x7fd8b40b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8ba301640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b40b3fa0 con 0x7fd8b40b7ba0 2024-03-16T20:45:14.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8ba301640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b40b50a0 con 0x7fd8b409ae30 2024-03-16T20:45:14.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8ba301640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b40b8590 con 0x7fd8b41429f0 2024-03-16T20:45:14.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8b3fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd8b409ae30 0x7fd8b40b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8b37fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fd8b40b7ba0 0x7fd8b41422e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58672/0 (socket says 172.21.15.122:58672) 2024-03-16T20:45:14.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8b37fe640 1 -- 172.21.15.122:0/2967882415 learned_addr learned my addr 172.21.15.122:0/2967882415 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:14.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8b889e640 1 --2- 172.21.15.122:0/2967882415 >> v2:172.21.15.195:3300/0 conn(0x7fd8b41429f0 0x7fd8b40b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8b3fff640 1 -- 172.21.15.122:0/2967882415 >> v2:172.21.15.195:3300/0 conn(0x7fd8b41429f0 msgr2=0x7fd8b40b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:14.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8b3fff640 1 --2- 172.21.15.122:0/2967882415 >> v2:172.21.15.195:3300/0 conn(0x7fd8b41429f0 0x7fd8b40b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8b3fff640 1 -- 172.21.15.122:0/2967882415 >> v1:172.21.15.122:6789/0 conn(0x7fd8b40b7ba0 legacy=0x7fd8b41422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8b3fff640 1 -- 172.21.15.122:0/2967882415 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b409b210 con 0x7fd8b409ae30 2024-03-16T20:45:14.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8b2ffd640 1 -- 172.21.15.122:0/2967882415 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3452759236 0 0) 0x7fd8b40b3fa0 con 0x7fd8b40b7ba0 2024-03-16T20:45:14.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8b3fff640 1 --2- 172.21.15.122:0/2967882415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b409ae30 0x7fd8b40b4430 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd8b406e0f0 tx=0x7fd8a401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9524909452902cf4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:14.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8b2ffd640 1 -- 172.21.15.122:0/2967882415 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40ab680 con 0x7fd8b409ae30 2024-03-16T20:45:14.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8b2ffd640 1 -- 172.21.15.122:0/2967882415 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a40ab7e0 con 0x7fd8b409ae30 2024-03-16T20:45:14.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8b2ffd640 1 -- 172.21.15.122:0/2967882415 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40abb70 con 0x7fd8b409ae30 2024-03-16T20:45:14.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8ba301640 1 -- 172.21.15.122:0/2967882415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b409ae30 msgr2=0x7fd8b40b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8ba301640 1 --2- 172.21.15.122:0/2967882415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b409ae30 0x7fd8b40b4430 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd8b406e0f0 tx=0x7fd8a401c480 comp rx=0 tx=0).stop 2024-03-16T20:45:14.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8ba301640 1 -- 172.21.15.122:0/2967882415 shutdown_connections 2024-03-16T20:45:14.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8ba301640 1 --2- 172.21.15.122:0/2967882415 >> v2:172.21.15.195:3300/0 conn(0x7fd8b41429f0 0x7fd8b40b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8ba301640 1 --2- 172.21.15.122:0/2967882415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b409ae30 0x7fd8b40b4430 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8ba301640 1 -- 172.21.15.122:0/2967882415 >> 172.21.15.122:0/2967882415 conn(0x7fd8b40b8d10 msgr2=0x7fd8b409a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:14.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8ba301640 1 -- 172.21.15.122:0/2967882415 shutdown_connections 2024-03-16T20:45:14.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.312+0000 7fd8ba301640 1 -- 172.21.15.122:0/2967882415 wait complete. 2024-03-16T20:45:14.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8ba301640 1 Processor -- start 2024-03-16T20:45:14.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8ba301640 1 -- start start 2024-03-16T20:45:14.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8ba301640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8b409ae30 0x7fd8b413f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8ba301640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8b40b7ba0 0x7fd8b41e2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8ba301640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b41429f0 0x7fd8b41e2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8ba301640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40b2810 con 0x7fd8b41429f0 2024-03-16T20:45:14.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8ba301640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40af780 con 0x7fd8b40b7ba0 2024-03-16T20:45:14.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8ba301640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b40af4a0 con 0x7fd8b409ae30 2024-03-16T20:45:14.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8b889e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b41429f0 0x7fd8b41e2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8b889e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b41429f0 0x7fd8b41e2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41712/0 (socket says 172.21.15.122:41712) 2024-03-16T20:45:14.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8b889e640 1 -- 172.21.15.122:0/3166113482 learned_addr learned my addr 172.21.15.122:0/3166113482 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:14.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8b3fff640 1 --2- 172.21.15.122:0/3166113482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8b409ae30 0x7fd8b413f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8b37fe640 1 --2- 172.21.15.122:0/3166113482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8b40b7ba0 0x7fd8b41e2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8b889e640 1 -- 172.21.15.122:0/3166113482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8b409ae30 msgr2=0x7fd8b413f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8b889e640 1 --2- 172.21.15.122:0/3166113482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8b409ae30 0x7fd8b413f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8b889e640 1 -- 172.21.15.122:0/3166113482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8b40b7ba0 msgr2=0x7fd8b41e2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8b889e640 1 --2- 172.21.15.122:0/3166113482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8b40b7ba0 0x7fd8b41e2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8b889e640 1 -- 172.21.15.122:0/3166113482 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a4009420 con 0x7fd8b41429f0 2024-03-16T20:45:14.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8b889e640 1 --2- 172.21.15.122:0/3166113482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b41429f0 0x7fd8b41e2fa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac007e60 tx=0x7fd8ac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:14.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8b0ff9640 1 -- 172.21.15.122:0/3166113482 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac014420 con 0x7fd8b41429f0 2024-03-16T20:45:14.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8b0ff9640 1 -- 172.21.15.122:0/3166113482 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac0145a0 con 0x7fd8b41429f0 2024-03-16T20:45:14.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8ba301640 1 -- 172.21.15.122:0/3166113482 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b413f580 con 0x7fd8b41429f0 2024-03-16T20:45:14.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8b0ff9640 1 -- 172.21.15.122:0/3166113482 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac014950 con 0x7fd8b41429f0 2024-03-16T20:45:14.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8ba301640 1 -- 172.21.15.122:0/3166113482 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b413f830 con 0x7fd8b41429f0 2024-03-16T20:45:14.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8ba301640 1 -- 172.21.15.122:0/3166113482 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b40adf80 con 0x7fd8b41429f0 2024-03-16T20:45:14.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8b0ff9640 1 -- 172.21.15.122:0/3166113482 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fd8ac014ab0 con 0x7fd8b41429f0 2024-03-16T20:45:14.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.316+0000 7fd8b0ff9640 1 --2- 172.21.15.122:0/3166113482 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd8a0066e40 0x7fd8a0069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.320+0000 7fd8b0ff9640 1 -- 172.21.15.122:0/3166113482 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8ac0852c0 con 0x7fd8b41429f0 2024-03-16T20:45:14.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.320+0000 7fd8b3fff640 1 --2- 172.21.15.122:0/3166113482 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd8a0066e40 0x7fd8a0069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.320+0000 7fd8b3fff640 1 --2- 172.21.15.122:0/3166113482 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd8a0066e40 0x7fd8a0069300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40af000 tx=0x7fd8a40a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:14.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.324+0000 7fd8b0ff9640 1 -- 172.21.15.122:0/3166113482 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd8b40adf80 con 0x7fd8b41429f0 2024-03-16T20:45:14.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.480+0000 7fd8ba301640 1 -- 172.21.15.122:0/3166113482 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd8b40adf80 con 0x7fd8b41429f0 2024-03-16T20:45:14.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.480+0000 7fd8b0ff9640 1 -- 172.21.15.122:0/3166113482 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd8b40adf80 con 0x7fd8b41429f0 2024-03-16T20:45:14.485 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-03-16T20:45:14.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.484+0000 7fd8ba301640 1 -- 172.21.15.122:0/3166113482 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd8a0066e40 msgr2=0x7fd8a0069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.484+0000 7fd8ba301640 1 --2- 172.21.15.122:0/3166113482 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd8a0066e40 0x7fd8a0069300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40af000 tx=0x7fd8a40a7000 comp rx=0 tx=0).stop 2024-03-16T20:45:14.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.484+0000 7fd8ba301640 1 -- 172.21.15.122:0/3166113482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b41429f0 msgr2=0x7fd8b41e2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.484+0000 7fd8ba301640 1 --2- 172.21.15.122:0/3166113482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b41429f0 0x7fd8b41e2fa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac007e60 tx=0x7fd8ac00e4b0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.484+0000 7fd8ba301640 1 -- 172.21.15.122:0/3166113482 shutdown_connections 2024-03-16T20:45:14.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.484+0000 7fd8ba301640 1 --2- 172.21.15.122:0/3166113482 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd8a0066e40 0x7fd8a0069300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.484+0000 7fd8ba301640 1 --2- 172.21.15.122:0/3166113482 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b41429f0 0x7fd8b41e2fa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.484+0000 7fd8ba301640 1 --2- 172.21.15.122:0/3166113482 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8b40b7ba0 0x7fd8b41e2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.484+0000 7fd8ba301640 1 --2- 172.21.15.122:0/3166113482 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8b409ae30 0x7fd8b413f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.484+0000 7fd8ba301640 1 -- 172.21.15.122:0/3166113482 >> 172.21.15.122:0/3166113482 conn(0x7fd8b40b8d10 msgr2=0x7fd8b409b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:14.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.484+0000 7fd8ba301640 1 -- 172.21.15.122:0/3166113482 shutdown_connections 2024-03-16T20:45:14.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.484+0000 7fd8ba301640 1 -- 172.21.15.122:0/3166113482 wait complete. 2024-03-16T20:45:14.504 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T20:45:14.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.616+0000 7f5d143e8640 1 Processor -- start 2024-03-16T20:45:14.621 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.616+0000 7f5d143e8640 1 -- start start 2024-03-16T20:45:14.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d143e8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d0c072760 0x7f5d0c0b56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d143e8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5d0c13ecf0 0x7f5d0c13f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.623 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d143e8640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d0c0b5210 con 0x7f5d0c0b3110 2024-03-16T20:45:14.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d143e8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0c06b5c0 con 0x7f5d0c13ecf0 2024-03-16T20:45:14.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d143e8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c0b48a0 con 0x7f5d0c072760 2024-03-16T20:45:14.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d12184640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f5d0c0b3110 0x7f5d0c144e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58680/0 (socket says 172.21.15.122:58680) 2024-03-16T20:45:14.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d12184640 1 -- 172.21.15.122:0/1705006190 learned_addr learned my addr 172.21.15.122:0/1705006190 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:14.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11983640 1 --2- 172.21.15.122:0/1705006190 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d0c072760 0x7f5d0c0b56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d12985640 1 --2- 172.21.15.122:0/1705006190 >> v2:172.21.15.195:3300/0 conn(0x7f5d0c13ecf0 0x7f5d0c13f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 -- 172.21.15.122:0/1705006190 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315856902 0 0) 0x7f5d0c0b5210 con 0x7f5d0c0b3110 2024-03-16T20:45:14.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 -- 172.21.15.122:0/1705006190 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cf8003530 con 0x7f5d0c0b3110 2024-03-16T20:45:14.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 -- 172.21.15.122:0/1705006190 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3052089736 0 0) 0x7f5cf8003530 con 0x7f5d0c0b3110 2024-03-16T20:45:14.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 -- 172.21.15.122:0/1705006190 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d0c072760 msgr2=0x7f5d0c0b56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:14.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 --2- 172.21.15.122:0/1705006190 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d0c072760 0x7f5d0c0b56a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 -- 172.21.15.122:0/1705006190 >> v2:172.21.15.195:3300/0 conn(0x7f5d0c13ecf0 msgr2=0x7f5d0c13f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:14.624 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 --2- 172.21.15.122:0/1705006190 >> v2:172.21.15.195:3300/0 conn(0x7f5d0c13ecf0 0x7f5d0c13f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 -- 172.21.15.122:0/1705006190 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c0b34f0 con 0x7f5d0c0b3110 2024-03-16T20:45:14.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 -- 172.21.15.122:0/1705006190 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f5cfc001680 con 0x7f5d0c0b3110 2024-03-16T20:45:14.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 -- 172.21.15.122:0/1705006190 >> v1:172.21.15.122:6789/0 conn(0x7f5d0c0b3110 legacy=0x7f5d0c144e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 --2- 172.21.15.122:0/1705006190 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d0c072760 0x7f5cf8003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 --2- 172.21.15.122:0/1705006190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf8003f00 0x7f5cf80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 --2- 172.21.15.122:0/1705006190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf8004d10 0x7f5cf8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 -- 172.21.15.122:0/1705006190 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf800b990 con 0x7f5cf8003f00 2024-03-16T20:45:14.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 -- 172.21.15.122:0/1705006190 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf800bad0 con 0x7f5cf8004d10 2024-03-16T20:45:14.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 -- 172.21.15.122:0/1705006190 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf800bc60 con 0x7f5d0c072760 2024-03-16T20:45:14.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11983640 1 --2- 172.21.15.122:0/1705006190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf8003f00 0x7f5cf80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d12184640 1 --2- 172.21.15.122:0/1705006190 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d0c072760 0x7f5cf8003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d12985640 1 --2- 172.21.15.122:0/1705006190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf8004d10 0x7f5cf8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.625 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11983640 1 -- 172.21.15.122:0/1705006190 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d0c072760 msgr2=0x7f5cf8003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:14.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11983640 1 --2- 172.21.15.122:0/1705006190 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d0c072760 0x7f5cf8003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11983640 1 -- 172.21.15.122:0/1705006190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf8004d10 msgr2=0x7f5cf8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:14.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11983640 1 --2- 172.21.15.122:0/1705006190 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf8004d10 0x7f5cf8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.626 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11983640 1 -- 172.21.15.122:0/1705006190 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf8003530 con 0x7f5cf8003f00 2024-03-16T20:45:14.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11983640 1 --2- 172.21.15.122:0/1705006190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf8003f00 0x7f5cf80047d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009b10 tx=0x7f5d0801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2cddfe469171817 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:14.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 -- 172.21.15.122:0/1705006190 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d080a6450 con 0x7f5cf8003f00 2024-03-16T20:45:14.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 -- 172.21.15.122:0/1705006190 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d080a65b0 con 0x7f5cf8003f00 2024-03-16T20:45:14.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.620+0000 7f5d11182640 1 -- 172.21.15.122:0/1705006190 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d080a6960 con 0x7f5cf8003f00 2024-03-16T20:45:14.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d143e8640 1 -- 172.21.15.122:0/1705006190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf8003f00 msgr2=0x7f5cf80047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d143e8640 1 --2- 172.21.15.122:0/1705006190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf8003f00 0x7f5cf80047d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009b10 tx=0x7f5d0801ca70 comp rx=0 tx=0).stop 2024-03-16T20:45:14.627 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d12985640 1 -- 172.21.15.122:0/1705006190 reap_dead start 2024-03-16T20:45:14.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d143e8640 1 -- 172.21.15.122:0/1705006190 shutdown_connections 2024-03-16T20:45:14.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d143e8640 1 -- 172.21.15.122:0/1705006190 >> 172.21.15.122:0/1705006190 conn(0x7f5d0c0b97a0 msgr2=0x7f5d0c0b9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:14.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d143e8640 1 -- 172.21.15.122:0/1705006190 shutdown_connections 2024-03-16T20:45:14.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d143e8640 1 -- 172.21.15.122:0/1705006190 wait complete. 2024-03-16T20:45:14.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d143e8640 1 Processor -- start 2024-03-16T20:45:14.628 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d143e8640 1 -- start start 2024-03-16T20:45:14.630 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d143e8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d0c14abe0 0x7f5d0c1dd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d143e8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d0c13fea0 0x7f5d0c13b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d143e8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d0c147ee0 0x7f5d0c13bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d143e8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c1486e0 con 0x7f5d0c13fea0 2024-03-16T20:45:14.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d143e8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c13fa10 con 0x7f5d0c14abe0 2024-03-16T20:45:14.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d143e8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0c14afc0 con 0x7f5d0c147ee0 2024-03-16T20:45:14.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d11983640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d0c13fea0 0x7f5d0c13b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d11983640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d0c13fea0 0x7f5d0c13b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41736/0 (socket says 172.21.15.122:41736) 2024-03-16T20:45:14.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d11983640 1 -- 172.21.15.122:0/2063735078 learned_addr learned my addr 172.21.15.122:0/2063735078 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:14.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d12985640 1 --2- 172.21.15.122:0/2063735078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d0c147ee0 0x7f5d0c13bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d12184640 1 --2- 172.21.15.122:0/2063735078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d0c14abe0 0x7f5d0c1dd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d11983640 1 -- 172.21.15.122:0/2063735078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d0c147ee0 msgr2=0x7f5d0c13bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d11983640 1 --2- 172.21.15.122:0/2063735078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d0c147ee0 0x7f5d0c13bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.631 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d11983640 1 -- 172.21.15.122:0/2063735078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d0c14abe0 msgr2=0x7f5d0c1dd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d11983640 1 --2- 172.21.15.122:0/2063735078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d0c14abe0 0x7f5d0c1dd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d11983640 1 -- 172.21.15.122:0/2063735078 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d0c147320 con 0x7f5d0c13fea0 2024-03-16T20:45:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d12184640 1 --2- 172.21.15.122:0/2063735078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d0c14abe0 0x7f5d0c1dd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d11983640 1 --2- 172.21.15.122:0/2063735078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d0c13fea0 0x7f5d0c13b870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009a00 tx=0x7f5d08019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d02ffd640 1 -- 172.21.15.122:0/2063735078 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0800f000 con 0x7f5d0c13fea0 2024-03-16T20:45:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d02ffd640 1 -- 172.21.15.122:0/2063735078 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d080a6d20 con 0x7f5d0c13fea0 2024-03-16T20:45:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d02ffd640 1 -- 172.21.15.122:0/2063735078 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d080a7670 con 0x7f5d0c13fea0 2024-03-16T20:45:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d143e8640 1 -- 172.21.15.122:0/2063735078 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0c148400 con 0x7f5d0c13fea0 2024-03-16T20:45:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.624+0000 7f5d143e8640 1 -- 172.21.15.122:0/2063735078 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0c13c2f0 con 0x7f5d0c13fea0 2024-03-16T20:45:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.628+0000 7f5d143e8640 1 -- 172.21.15.122:0/2063735078 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0c13fbc0 con 0x7f5d0c13fea0 2024-03-16T20:45:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.628+0000 7f5d02ffd640 1 -- 172.21.15.122:0/2063735078 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f5d080a77d0 con 0x7f5d0c13fea0 2024-03-16T20:45:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.628+0000 7f5d02ffd640 1 --2- 172.21.15.122:0/2063735078 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5cf0066e70 0x7f5cf0069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.628+0000 7f5d02ffd640 1 -- 172.21.15.122:0/2063735078 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d0811e120 con 0x7f5d0c13fea0 2024-03-16T20:45:14.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.628+0000 7f5d12184640 1 --2- 172.21.15.122:0/2063735078 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5cf0066e70 0x7f5cf0069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.633 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.628+0000 7f5d12184640 1 --2- 172.21.15.122:0/2063735078 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5cf0066e70 0x7f5cf0069330 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc001ec0 tx=0x7f5cfc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:14.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.632+0000 7f5d02ffd640 1 -- 172.21.15.122:0/2063735078 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5d0c13fbc0 con 0x7f5d0c13fea0 2024-03-16T20:45:14.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.812+0000 7f5d143e8640 1 -- 172.21.15.122:0/2063735078 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5d0c13fbc0 con 0x7f5d0c13fea0 2024-03-16T20:45:14.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.812+0000 7f5d02ffd640 1 -- 172.21.15.122:0/2063735078 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3397 (secure 0 0 0) 0x7f5d080bed60 con 0x7f5d0c13fea0 2024-03-16T20:45:14.815 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:14.815 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3481092793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3481092793},{"type":"v1","addr":"172.21.15.122:6833","nonce":3481092793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T20:45:04.711887+0000","modified":"2024-03-16T20:45:06.765685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/1428608180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1428608180},{"type":"v1","addr":"172.21.15.195:6835","nonce":1428608180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-03-16T20:45:14.815 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 6 2024-03-16T20:45:14.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.812+0000 7f5d143e8640 1 -- 172.21.15.122:0/2063735078 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5cf0066e70 msgr2=0x7f5cf0069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.812+0000 7f5d143e8640 1 --2- 172.21.15.122:0/2063735078 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5cf0066e70 0x7f5cf0069330 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc001ec0 tx=0x7f5cfc009040 comp rx=0 tx=0).stop 2024-03-16T20:45:14.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.812+0000 7f5d143e8640 1 -- 172.21.15.122:0/2063735078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d0c13fea0 msgr2=0x7f5d0c13b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.812+0000 7f5d143e8640 1 --2- 172.21.15.122:0/2063735078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d0c13fea0 0x7f5d0c13b870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5d08009a00 tx=0x7f5d08019810 comp rx=0 tx=0).stop 2024-03-16T20:45:14.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.812+0000 7f5d143e8640 1 -- 172.21.15.122:0/2063735078 shutdown_connections 2024-03-16T20:45:14.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.812+0000 7f5d143e8640 1 --2- 172.21.15.122:0/2063735078 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5cf0066e70 0x7f5cf0069330 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.812+0000 7f5d143e8640 1 --2- 172.21.15.122:0/2063735078 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d0c147ee0 0x7f5d0c13bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.812+0000 7f5d143e8640 1 --2- 172.21.15.122:0/2063735078 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d0c13fea0 0x7f5d0c13b870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.812+0000 7f5d143e8640 1 --2- 172.21.15.122:0/2063735078 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d0c14abe0 0x7f5d0c1dd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.812+0000 7f5d143e8640 1 -- 172.21.15.122:0/2063735078 >> 172.21.15.122:0/2063735078 conn(0x7f5d0c0b97a0 msgr2=0x7f5d0c06c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:14.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.812+0000 7f5d143e8640 1 -- 172.21.15.122:0/2063735078 shutdown_connections 2024-03-16T20:45:14.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.812+0000 7f5d143e8640 1 -- 172.21.15.122:0/2063735078 wait complete. 2024-03-16T20:45:14.833 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:14.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.944+0000 7ff8bb768640 1 Processor -- start 2024-03-16T20:45:14.949 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.944+0000 7ff8bb768640 1 -- start start 2024-03-16T20:45:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8bb768640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff8b4072120 0x7ff8b40afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8bb768640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b40b1050 con 0x7ff8b4148b40 2024-03-16T20:45:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8bb768640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8b4072500 con 0x7ff8b413ed60 2024-03-16T20:45:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8bb768640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8b40b38a0 con 0x7ff8b4072120 2024-03-16T20:45:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8b9504640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff8b4072120 0x7ff8b40afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8b9d05640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7ff8b4148b40 0x7ff8b414f780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58686/0 (socket says 172.21.15.122:58686) 2024-03-16T20:45:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8b9d05640 1 -- 172.21.15.122:0/2802218026 learned_addr learned my addr 172.21.15.122:0/2802218026 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8b9504640 1 -- 172.21.15.122:0/2802218026 >> v1:172.21.15.195:6789/0 conn(0x7ff8b413ed60 legacy=0x7ff8b4148430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8b9504640 1 -- 172.21.15.122:0/2802218026 >> v1:172.21.15.122:6789/0 conn(0x7ff8b4148b40 legacy=0x7ff8b414f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.950 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8b9504640 1 -- 172.21.15.122:0/2802218026 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b40b3b80 con 0x7ff8b4072120 2024-03-16T20:45:14.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8b9504640 1 --2- 172.21.15.122:0/2802218026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b4072120 0x7ff8b40afb90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff8a40097c0 tx=0x7ff8a401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389a9c787ac3a1b1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:14.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8abfff640 1 -- 172.21.15.122:0/2802218026 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a40ac640 con 0x7ff8b4072120 2024-03-16T20:45:14.952 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8abfff640 1 -- 172.21.15.122:0/2802218026 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a40ac7a0 con 0x7ff8b4072120 2024-03-16T20:45:14.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8abfff640 1 -- 172.21.15.122:0/2802218026 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a40acb30 con 0x7ff8b4072120 2024-03-16T20:45:14.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8bb768640 1 -- 172.21.15.122:0/2802218026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b4072120 msgr2=0x7ff8b40afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8bb768640 1 --2- 172.21.15.122:0/2802218026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b4072120 0x7ff8b40afb90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff8a40097c0 tx=0x7ff8a401c3e0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8bb768640 1 -- 172.21.15.122:0/2802218026 shutdown_connections 2024-03-16T20:45:14.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8bb768640 1 --2- 172.21.15.122:0/2802218026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b4072120 0x7ff8b40afb90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.953 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8bb768640 1 -- 172.21.15.122:0/2802218026 >> 172.21.15.122:0/2802218026 conn(0x7ff8b40b8860 msgr2=0x7ff8b40b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:14.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8bb768640 1 -- 172.21.15.122:0/2802218026 shutdown_connections 2024-03-16T20:45:14.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.948+0000 7ff8bb768640 1 -- 172.21.15.122:0/2802218026 wait complete. 2024-03-16T20:45:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8bb768640 1 Processor -- start 2024-03-16T20:45:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8bb768640 1 -- start start 2024-03-16T20:45:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8bb768640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8b4072120 0x7ff8b41423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8bb768640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b413ed60 0x7ff8b4140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8bb768640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8b41413d0 0x7ff8b41dd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8bb768640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b413c100 con 0x7ff8b413ed60 2024-03-16T20:45:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8bb768640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b40af740 con 0x7ff8b4072120 2024-03-16T20:45:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8bb768640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b4141980 con 0x7ff8b41413d0 2024-03-16T20:45:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8b8d03640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b413ed60 0x7ff8b4140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8b8d03640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b413ed60 0x7ff8b4140e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41746/0 (socket says 172.21.15.122:41746) 2024-03-16T20:45:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8b8d03640 1 -- 172.21.15.122:0/304940011 learned_addr learned my addr 172.21.15.122:0/304940011 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:14.956 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8b9d05640 1 --2- 172.21.15.122:0/304940011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8b41413d0 0x7ff8b41dd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8b9504640 1 --2- 172.21.15.122:0/304940011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8b4072120 0x7ff8b41423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8b8d03640 1 -- 172.21.15.122:0/304940011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8b41413d0 msgr2=0x7ff8b41dd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8b8d03640 1 --2- 172.21.15.122:0/304940011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8b41413d0 0x7ff8b41dd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8b8d03640 1 -- 172.21.15.122:0/304940011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8b4072120 msgr2=0x7ff8b41423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:14.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8b8d03640 1 --2- 172.21.15.122:0/304940011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8b4072120 0x7ff8b41423e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:14.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8b8d03640 1 -- 172.21.15.122:0/304940011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a4009420 con 0x7ff8b413ed60 2024-03-16T20:45:14.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8b9504640 1 --2- 172.21.15.122:0/304940011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8b4072120 0x7ff8b41423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:14.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8b8d03640 1 --2- 172.21.15.122:0/304940011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b413ed60 0x7ff8b4140e90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000c420 tx=0x7ff8b000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:14.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8a9ffb640 1 -- 172.21.15.122:0/304940011 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b00052d0 con 0x7ff8b413ed60 2024-03-16T20:45:14.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8a9ffb640 1 -- 172.21.15.122:0/304940011 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b0005430 con 0x7ff8b413ed60 2024-03-16T20:45:14.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8bb768640 1 -- 172.21.15.122:0/304940011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b413ba40 con 0x7ff8b413ed60 2024-03-16T20:45:14.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8a9ffb640 1 -- 172.21.15.122:0/304940011 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0007260 con 0x7ff8b413ed60 2024-03-16T20:45:14.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8bb768640 1 -- 172.21.15.122:0/304940011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b41ddd00 con 0x7ff8b413ed60 2024-03-16T20:45:14.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8bb768640 1 -- 172.21.15.122:0/304940011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b413c240 con 0x7ff8b413ed60 2024-03-16T20:45:14.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.952+0000 7ff8a9ffb640 1 -- 172.21.15.122:0/304940011 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7ff8b00073c0 con 0x7ff8b413ed60 2024-03-16T20:45:14.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.956+0000 7ff8a9ffb640 1 --2- 172.21.15.122:0/304940011 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff898066e40 0x7ff898069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:14.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.956+0000 7ff8a9ffb640 1 -- 172.21.15.122:0/304940011 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8b0083dd0 con 0x7ff8b413ed60 2024-03-16T20:45:14.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.956+0000 7ff8b9504640 1 --2- 172.21.15.122:0/304940011 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff898066e40 0x7ff898069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:14.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.956+0000 7ff8b9504640 1 --2- 172.21.15.122:0/304940011 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff898066e40 0x7ff898069300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff8a40ae000 tx=0x7ff8a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:14.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:14.960+0000 7ff8a9ffb640 1 -- 172.21.15.122:0/304940011 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff8b413c240 con 0x7ff8b413ed60 2024-03-16T20:45:15.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.120+0000 7ff8bb768640 1 -- 172.21.15.122:0/304940011 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8b413c240 con 0x7ff8b413ed60 2024-03-16T20:45:15.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.120+0000 7ff8a9ffb640 1 -- 172.21.15.122:0/304940011 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7ff8b413c240 con 0x7ff8b413ed60 2024-03-16T20:45:15.125 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:15.126 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:06.404479+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:15.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.124+0000 7ff8bb768640 1 -- 172.21.15.122:0/304940011 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff898066e40 msgr2=0x7ff898069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.124+0000 7ff8bb768640 1 --2- 172.21.15.122:0/304940011 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff898066e40 0x7ff898069300 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff8a40ae000 tx=0x7ff8a4008040 comp rx=0 tx=0).stop 2024-03-16T20:45:15.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.124+0000 7ff8bb768640 1 -- 172.21.15.122:0/304940011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b413ed60 msgr2=0x7ff8b4140e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.124+0000 7ff8bb768640 1 --2- 172.21.15.122:0/304940011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b413ed60 0x7ff8b4140e90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff8b000c420 tx=0x7ff8b000c8e0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.124+0000 7ff8bb768640 1 -- 172.21.15.122:0/304940011 shutdown_connections 2024-03-16T20:45:15.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.124+0000 7ff8bb768640 1 --2- 172.21.15.122:0/304940011 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff898066e40 0x7ff898069300 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.124+0000 7ff8bb768640 1 --2- 172.21.15.122:0/304940011 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff8b41413d0 0x7ff8b41dd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.124+0000 7ff8bb768640 1 --2- 172.21.15.122:0/304940011 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff8b413ed60 0x7ff8b4140e90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.124+0000 7ff8bb768640 1 --2- 172.21.15.122:0/304940011 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff8b4072120 0x7ff8b41423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.124+0000 7ff8bb768640 1 -- 172.21.15.122:0/304940011 >> 172.21.15.122:0/304940011 conn(0x7ff8b40b8860 msgr2=0x7ff8b406b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:15.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.124+0000 7ff8bb768640 1 -- 172.21.15.122:0/304940011 shutdown_connections 2024-03-16T20:45:15.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.124+0000 7ff8bb768640 1 -- 172.21.15.122:0/304940011 wait complete. 2024-03-16T20:45:15.145 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T20:44:52.653523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T20:45:02.758515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T20:45:03.710783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T20:45:15.145 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T20:45:15.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc3c952640 1 Processor -- start 2024-03-16T20:45:15.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc3c952640 1 -- start start 2024-03-16T20:45:15.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc3c952640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc3809ae30 0x7efc380b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc3c952640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7efc380b7ba0 0x7efc380b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc3c952640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc380b3fa0 con 0x7efc381422e0 2024-03-16T20:45:15.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc3c952640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc380b50a0 con 0x7efc380b7ba0 2024-03-16T20:45:15.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc3c952640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc380b8560 con 0x7efc3809ae30 2024-03-16T20:45:15.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc36d9d640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7efc381422e0 0x7efc38142730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58688/0 (socket says 172.21.15.122:58688) 2024-03-16T20:45:15.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc36d9d640 1 -- 172.21.15.122:0/3501438160 learned_addr learned my addr 172.21.15.122:0/3501438160 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:15.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc35d9b640 1 --2- 172.21.15.122:0/3501438160 >> v2:172.21.15.195:3300/0 conn(0x7efc380b7ba0 0x7efc380b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc3659c640 1 --2- 172.21.15.122:0/3501438160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc3809ae30 0x7efc380b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc3559a640 1 -- 172.21.15.122:0/3501438160 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943555754 0 0) 0x7efc380b3fa0 con 0x7efc381422e0 2024-03-16T20:45:15.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc3559a640 1 -- 172.21.15.122:0/3501438160 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc20003530 con 0x7efc381422e0 2024-03-16T20:45:15.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc3559a640 1 -- 172.21.15.122:0/3501438160 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2114345560 0 0) 0x7efc20003530 con 0x7efc381422e0 2024-03-16T20:45:15.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc3559a640 1 -- 172.21.15.122:0/3501438160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc3809ae30 msgr2=0x7efc380b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:15.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc3559a640 1 --2- 172.21.15.122:0/3501438160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc3809ae30 0x7efc380b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc3559a640 1 -- 172.21.15.122:0/3501438160 >> v2:172.21.15.195:3300/0 conn(0x7efc380b7ba0 msgr2=0x7efc380b8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:15.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc3559a640 1 --2- 172.21.15.122:0/3501438160 >> v2:172.21.15.195:3300/0 conn(0x7efc380b7ba0 0x7efc380b8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc3559a640 1 -- 172.21.15.122:0/3501438160 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3809b210 con 0x7efc381422e0 2024-03-16T20:45:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.264+0000 7efc3559a640 1 -- 172.21.15.122:0/3501438160 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7efc30001a20 con 0x7efc381422e0 2024-03-16T20:45:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc3559a640 1 -- 172.21.15.122:0/3501438160 >> v1:172.21.15.122:6789/0 conn(0x7efc381422e0 legacy=0x7efc38142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc3559a640 1 --2- 172.21.15.122:0/3501438160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc20003980 0x7efc20003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc3559a640 1 --2- 172.21.15.122:0/3501438160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc200047d0 0x7efc20004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc3559a640 1 --2- 172.21.15.122:0/3501438160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc3809ae30 0x7efc20005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc3559a640 1 -- 172.21.15.122:0/3501438160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc380b50a0 con 0x7efc200047d0 2024-03-16T20:45:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc3559a640 1 -- 172.21.15.122:0/3501438160 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc380b8560 con 0x7efc20003980 2024-03-16T20:45:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc3559a640 1 -- 172.21.15.122:0/3501438160 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2000b990 con 0x7efc3809ae30 2024-03-16T20:45:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc35d9b640 1 --2- 172.21.15.122:0/3501438160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc200047d0 0x7efc20004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc3659c640 1 --2- 172.21.15.122:0/3501438160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc20003980 0x7efc20003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc36d9d640 1 --2- 172.21.15.122:0/3501438160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc3809ae30 0x7efc20005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc35d9b640 1 -- 172.21.15.122:0/3501438160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc3809ae30 msgr2=0x7efc20005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc35d9b640 1 --2- 172.21.15.122:0/3501438160 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc3809ae30 0x7efc20005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc35d9b640 1 -- 172.21.15.122:0/3501438160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc20003980 msgr2=0x7efc20003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc35d9b640 1 --2- 172.21.15.122:0/3501438160 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc20003980 0x7efc20003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc35d9b640 1 -- 172.21.15.122:0/3501438160 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc20003530 con 0x7efc200047d0 2024-03-16T20:45:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc35d9b640 1 --2- 172.21.15.122:0/3501438160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc200047d0 0x7efc20004c50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efc2c009ae0 tx=0x7efc2c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c53a37b7cd5d2e8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc3559a640 1 -- 172.21.15.122:0/3501438160 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c0a55c0 con 0x7efc200047d0 2024-03-16T20:45:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc3559a640 1 -- 172.21.15.122:0/3501438160 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2c0a5720 con 0x7efc200047d0 2024-03-16T20:45:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc3559a640 1 -- 172.21.15.122:0/3501438160 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c0a5ad0 con 0x7efc200047d0 2024-03-16T20:45:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc3c952640 1 -- 172.21.15.122:0/3501438160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc200047d0 msgr2=0x7efc20004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc3c952640 1 --2- 172.21.15.122:0/3501438160 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc200047d0 0x7efc20004c50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efc2c009ae0 tx=0x7efc2c01cab0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc36d9d640 1 -- 172.21.15.122:0/3501438160 reap_dead start 2024-03-16T20:45:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc3c952640 1 -- 172.21.15.122:0/3501438160 shutdown_connections 2024-03-16T20:45:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc3c952640 1 -- 172.21.15.122:0/3501438160 >> 172.21.15.122:0/3501438160 conn(0x7efc380b8d10 msgr2=0x7efc3809a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc3c952640 1 -- 172.21.15.122:0/3501438160 shutdown_connections 2024-03-16T20:45:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.268+0000 7efc3c952640 1 -- 172.21.15.122:0/3501438160 wait complete. 2024-03-16T20:45:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc3c952640 1 Processor -- start 2024-03-16T20:45:15.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc3c952640 1 -- start start 2024-03-16T20:45:15.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc3c952640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc200047d0 0x7efc380b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc3c952640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc381da270 0x7efc381da6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc3c952640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc381458a0 0x7efc38145d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc3c952640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc38142ea0 con 0x7efc381458a0 2024-03-16T20:45:15.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc3c952640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc381de280 con 0x7efc381da270 2024-03-16T20:45:15.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc3c952640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc38143ed0 con 0x7efc200047d0 2024-03-16T20:45:15.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc36d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc381458a0 0x7efc38145d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc36d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc381458a0 0x7efc38145d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41754/0 (socket says 172.21.15.122:41754) 2024-03-16T20:45:15.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc36d9d640 1 -- 172.21.15.122:0/3647940085 learned_addr learned my addr 172.21.15.122:0/3647940085 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:15.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc3659c640 1 --2- 172.21.15.122:0/3647940085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc200047d0 0x7efc380b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc35d9b640 1 --2- 172.21.15.122:0/3647940085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc381da270 0x7efc381da6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc36d9d640 1 -- 172.21.15.122:0/3647940085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc200047d0 msgr2=0x7efc380b2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc36d9d640 1 --2- 172.21.15.122:0/3647940085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc200047d0 0x7efc380b2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc36d9d640 1 -- 172.21.15.122:0/3647940085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc381da270 msgr2=0x7efc381da6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc36d9d640 1 --2- 172.21.15.122:0/3647940085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc381da270 0x7efc381da6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc36d9d640 1 -- 172.21.15.122:0/3647940085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc38146260 con 0x7efc381458a0 2024-03-16T20:45:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc36d9d640 1 --2- 172.21.15.122:0/3647940085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc381458a0 0x7efc38145d20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efc380b4130 tx=0x7efc30005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc26ffd640 1 -- 172.21.15.122:0/3647940085 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3000eb50 con 0x7efc381458a0 2024-03-16T20:45:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc26ffd640 1 -- 172.21.15.122:0/3647940085 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3000ecb0 con 0x7efc381458a0 2024-03-16T20:45:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc3c952640 1 -- 172.21.15.122:0/3647940085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc381430f0 con 0x7efc381458a0 2024-03-16T20:45:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc26ffd640 1 -- 172.21.15.122:0/3647940085 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30015400 con 0x7efc381458a0 2024-03-16T20:45:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc3c952640 1 -- 172.21.15.122:0/3647940085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc381ddf10 con 0x7efc381458a0 2024-03-16T20:45:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc3c952640 1 -- 172.21.15.122:0/3647940085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc381ddd40 con 0x7efc381458a0 2024-03-16T20:45:15.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc26ffd640 1 -- 172.21.15.122:0/3647940085 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7efc300143a0 con 0x7efc381458a0 2024-03-16T20:45:15.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc26ffd640 1 --2- 172.21.15.122:0/3647940085 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efc18066e70 0x7efc18069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc26ffd640 1 -- 172.21.15.122:0/3647940085 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc30087560 con 0x7efc381458a0 2024-03-16T20:45:15.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.272+0000 7efc3659c640 1 --2- 172.21.15.122:0/3647940085 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efc18066e70 0x7efc18069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.276+0000 7efc3659c640 1 --2- 172.21.15.122:0/3647940085 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efc18066e70 0x7efc18069330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efc28009950 tx=0x7efc28008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:15.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.280+0000 7efc26ffd640 1 -- 172.21.15.122:0/3647940085 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7efc300543e0 con 0x7efc381458a0 2024-03-16T20:45:15.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.424+0000 7efc3c952640 1 -- 172.21.15.122:0/3647940085 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc38074bb0 con 0x7efc381458a0 2024-03-16T20:45:15.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.424+0000 7efc26ffd640 1 -- 172.21.15.122:0/3647940085 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc3001a0a0 con 0x7efc381458a0 2024-03-16T20:45:15.427 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-03-16T20:45:15.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.424+0000 7efc3c952640 1 -- 172.21.15.122:0/3647940085 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efc18066e70 msgr2=0x7efc18069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.424+0000 7efc3c952640 1 --2- 172.21.15.122:0/3647940085 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efc18066e70 0x7efc18069330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efc28009950 tx=0x7efc28008040 comp rx=0 tx=0).stop 2024-03-16T20:45:15.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.424+0000 7efc3c952640 1 -- 172.21.15.122:0/3647940085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc381458a0 msgr2=0x7efc38145d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.424+0000 7efc3c952640 1 --2- 172.21.15.122:0/3647940085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc381458a0 0x7efc38145d20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efc380b4130 tx=0x7efc30005a80 comp rx=0 tx=0).stop 2024-03-16T20:45:15.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.424+0000 7efc3c952640 1 -- 172.21.15.122:0/3647940085 shutdown_connections 2024-03-16T20:45:15.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.424+0000 7efc3c952640 1 --2- 172.21.15.122:0/3647940085 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efc18066e70 0x7efc18069330 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.424+0000 7efc3c952640 1 --2- 172.21.15.122:0/3647940085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc381458a0 0x7efc38145d20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.424+0000 7efc3c952640 1 --2- 172.21.15.122:0/3647940085 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efc381da270 0x7efc381da6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.424+0000 7efc3c952640 1 --2- 172.21.15.122:0/3647940085 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efc200047d0 0x7efc380b2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.424+0000 7efc3c952640 1 -- 172.21.15.122:0/3647940085 >> 172.21.15.122:0/3647940085 conn(0x7efc380b8d10 msgr2=0x7efc38074320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:15.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.424+0000 7efc3c952640 1 -- 172.21.15.122:0/3647940085 shutdown_connections 2024-03-16T20:45:15.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.424+0000 7efc3c952640 1 -- 172.21.15.122:0/3647940085 wait complete. 2024-03-16T20:45:15.444 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T20:45:15.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.552+0000 7f5d42c8f640 1 Processor -- start 2024-03-16T20:45:15.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.552+0000 7f5d42c8f640 1 -- start start 2024-03-16T20:45:15.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d42c8f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5d3c06d350 0x7f5d3c152930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d42c8f640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c147ff0 con 0x7f5d3c0bb1d0 2024-03-16T20:45:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d42c8f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c1479a0 con 0x7f5d3c141580 2024-03-16T20:45:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d42c8f640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d3c147c80 con 0x7f5d3c06d350 2024-03-16T20:45:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d4122c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5d3c06d350 0x7f5d3c152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d3bfff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f5d3c0bb1d0 0x7f5d3c06cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58700/0 (socket says 172.21.15.122:58700) 2024-03-16T20:45:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d3bfff640 1 -- 172.21.15.122:0/2164175772 learned_addr learned my addr 172.21.15.122:0/2164175772 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d3b7fe640 1 -- 172.21.15.122:0/2164175772 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351241931 0 0) 0x7f5d3c147ff0 con 0x7f5d3c0bb1d0 2024-03-16T20:45:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d3b7fe640 1 -- 172.21.15.122:0/2164175772 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d24003530 con 0x7f5d3c0bb1d0 2024-03-16T20:45:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d4122c640 1 -- 172.21.15.122:0/2164175772 >> v1:172.21.15.195:6789/0 conn(0x7f5d3c141580 legacy=0x7f5d3c0baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d4122c640 1 -- 172.21.15.122:0/2164175772 >> v1:172.21.15.122:6789/0 conn(0x7f5d3c0bb1d0 legacy=0x7f5d3c06cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d4122c640 1 -- 172.21.15.122:0/2164175772 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c06d7e0 con 0x7f5d3c06d350 2024-03-16T20:45:15.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d4122c640 1 --2- 172.21.15.122:0/2164175772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d3c06d350 0x7f5d3c152930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009d90 tx=0x7f5d3401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7716dca29f84b0df server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:15.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d3b7fe640 1 -- 172.21.15.122:0/2164175772 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d340a5830 con 0x7f5d3c06d350 2024-03-16T20:45:15.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d3b7fe640 1 -- 172.21.15.122:0/2164175772 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d340a5990 con 0x7f5d3c06d350 2024-03-16T20:45:15.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d3b7fe640 1 -- 172.21.15.122:0/2164175772 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d340a5d20 con 0x7f5d3c06d350 2024-03-16T20:45:15.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d42c8f640 1 -- 172.21.15.122:0/2164175772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d3c06d350 msgr2=0x7f5d3c152930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d42c8f640 1 --2- 172.21.15.122:0/2164175772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d3c06d350 0x7f5d3c152930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009d90 tx=0x7f5d3401c970 comp rx=0 tx=0).stop 2024-03-16T20:45:15.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d42c8f640 1 -- 172.21.15.122:0/2164175772 shutdown_connections 2024-03-16T20:45:15.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d42c8f640 1 --2- 172.21.15.122:0/2164175772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d3c06d350 0x7f5d3c152930 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d42c8f640 1 -- 172.21.15.122:0/2164175772 >> 172.21.15.122:0/2164175772 conn(0x7f5d3c0bca20 msgr2=0x7f5d3c0bce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:15.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d42c8f640 1 -- 172.21.15.122:0/2164175772 shutdown_connections 2024-03-16T20:45:15.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.556+0000 7f5d42c8f640 1 -- 172.21.15.122:0/2164175772 wait complete. 2024-03-16T20:45:15.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d42c8f640 1 Processor -- start 2024-03-16T20:45:15.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d42c8f640 1 -- start start 2024-03-16T20:45:15.564 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d42c8f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d3c06d350 0x7f5d3c13bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d42c8f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d3c0bb1d0 0x7f5d3c0b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d42c8f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d3c141580 0x7f5d3c0b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d42c8f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c0b4d70 con 0x7f5d3c141580 2024-03-16T20:45:15.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d42c8f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c13e360 con 0x7f5d3c0bb1d0 2024-03-16T20:45:15.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d42c8f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c146e60 con 0x7f5d3c06d350 2024-03-16T20:45:15.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d4122c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d3c141580 0x7f5d3c0b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d4122c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d3c141580 0x7f5d3c0b5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41766/0 (socket says 172.21.15.122:41766) 2024-03-16T20:45:15.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d4122c640 1 -- 172.21.15.122:0/2624072135 learned_addr learned my addr 172.21.15.122:0/2624072135 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:15.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d3bfff640 1 --2- 172.21.15.122:0/2624072135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d3c0bb1d0 0x7f5d3c0b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d40a2b640 1 --2- 172.21.15.122:0/2624072135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d3c06d350 0x7f5d3c13bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d4122c640 1 -- 172.21.15.122:0/2624072135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d3c06d350 msgr2=0x7f5d3c13bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d4122c640 1 --2- 172.21.15.122:0/2624072135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d3c06d350 0x7f5d3c13bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d4122c640 1 -- 172.21.15.122:0/2624072135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d3c0bb1d0 msgr2=0x7f5d3c0b57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d4122c640 1 --2- 172.21.15.122:0/2624072135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d3c0bb1d0 0x7f5d3c0b57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.565 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d4122c640 1 -- 172.21.15.122:0/2624072135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d34009a70 con 0x7f5d3c141580 2024-03-16T20:45:15.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d3bfff640 1 --2- 172.21.15.122:0/2624072135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d3c0bb1d0 0x7f5d3c0b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:15.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d4122c640 1 --2- 172.21.15.122:0/2624072135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d3c141580 0x7f5d3c0b5d00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5d3401a5a0 tx=0x7f5d34019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:15.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d397fa640 1 -- 172.21.15.122:0/2624072135 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d340ad640 con 0x7f5d3c141580 2024-03-16T20:45:15.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d397fa640 1 -- 172.21.15.122:0/2624072135 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d340ad7a0 con 0x7f5d3c141580 2024-03-16T20:45:15.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d42c8f640 1 -- 172.21.15.122:0/2624072135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3c144910 con 0x7f5d3c141580 2024-03-16T20:45:15.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d397fa640 1 -- 172.21.15.122:0/2624072135 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d340adb10 con 0x7f5d3c141580 2024-03-16T20:45:15.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d42c8f640 1 -- 172.21.15.122:0/2624072135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3c0b52a0 con 0x7f5d3c141580 2024-03-16T20:45:15.566 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d42c8f640 1 -- 172.21.15.122:0/2624072135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d3c0b5490 con 0x7f5d3c141580 2024-03-16T20:45:15.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d397fa640 1 -- 172.21.15.122:0/2624072135 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f5d340b4050 con 0x7f5d3c141580 2024-03-16T20:45:15.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.560+0000 7f5d397fa640 1 --2- 172.21.15.122:0/2624072135 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5d28066e40 0x7f5d28069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.564+0000 7f5d397fa640 1 -- 172.21.15.122:0/2624072135 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d3411f6f0 con 0x7f5d3c141580 2024-03-16T20:45:15.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.564+0000 7f5d40a2b640 1 --2- 172.21.15.122:0/2624072135 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5d28066e40 0x7f5d28069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.564+0000 7f5d40a2b640 1 --2- 172.21.15.122:0/2624072135 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5d28066e40 0x7f5d28069300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0015b0 tx=0x7f5d2c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:15.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.568+0000 7f5d397fa640 1 -- 172.21.15.122:0/2624072135 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5d340ec600 con 0x7f5d3c141580 2024-03-16T20:45:15.715 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.712+0000 7f5d42c8f640 1 -- 172.21.15.122:0/2624072135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5d3c074bb0 con 0x7f5d3c141580 2024-03-16T20:45:15.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.712+0000 7f5d397fa640 1 -- 172.21.15.122:0/2624072135 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5d340efcc0 con 0x7f5d3c141580 2024-03-16T20:45:15.717 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-03-16T20:45:15.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.716+0000 7f5d42c8f640 1 -- 172.21.15.122:0/2624072135 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5d28066e40 msgr2=0x7f5d28069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.716+0000 7f5d42c8f640 1 --2- 172.21.15.122:0/2624072135 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5d28066e40 0x7f5d28069300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0015b0 tx=0x7f5d2c009040 comp rx=0 tx=0).stop 2024-03-16T20:45:15.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.716+0000 7f5d42c8f640 1 -- 172.21.15.122:0/2624072135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d3c141580 msgr2=0x7f5d3c0b5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.716+0000 7f5d42c8f640 1 --2- 172.21.15.122:0/2624072135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d3c141580 0x7f5d3c0b5d00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5d3401a5a0 tx=0x7f5d34019710 comp rx=0 tx=0).stop 2024-03-16T20:45:15.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.716+0000 7f5d42c8f640 1 -- 172.21.15.122:0/2624072135 shutdown_connections 2024-03-16T20:45:15.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.716+0000 7f5d42c8f640 1 --2- 172.21.15.122:0/2624072135 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5d28066e40 0x7f5d28069300 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.716+0000 7f5d42c8f640 1 --2- 172.21.15.122:0/2624072135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5d3c141580 0x7f5d3c0b5d00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.716+0000 7f5d42c8f640 1 --2- 172.21.15.122:0/2624072135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5d3c0bb1d0 0x7f5d3c0b57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.716+0000 7f5d42c8f640 1 --2- 172.21.15.122:0/2624072135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5d3c06d350 0x7f5d3c13bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.716+0000 7f5d42c8f640 1 -- 172.21.15.122:0/2624072135 >> 172.21.15.122:0/2624072135 conn(0x7f5d3c0bca20 msgr2=0x7f5d3c0b7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:15.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.716+0000 7f5d42c8f640 1 -- 172.21.15.122:0/2624072135 shutdown_connections 2024-03-16T20:45:15.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.716+0000 7f5d42c8f640 1 -- 172.21.15.122:0/2624072135 wait complete. 2024-03-16T20:45:15.734 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T20:45:15.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.852+0000 7fd63ed90640 1 Processor -- start 2024-03-16T20:45:15.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.852+0000 7fd63ed90640 1 -- start start 2024-03-16T20:45:15.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.852+0000 7fd63ed90640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd6380b7d40 0x7fd6380b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.852+0000 7fd63ed90640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd638148480 0x7fd63814f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.852+0000 7fd63ed90640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6380b6740 con 0x7fd6380b7540 2024-03-16T20:45:15.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.852+0000 7fd63ed90640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6380b23d0 con 0x7fd6380b7d40 2024-03-16T20:45:15.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.852+0000 7fd63ed90640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6380b7110 con 0x7fd638148480 2024-03-16T20:45:15.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.852+0000 7fd637fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd6380b7d40 0x7fd6380b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.852+0000 7fd63cb2c640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fd6380b7540 0x7fd63813ef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58714/0 (socket says 172.21.15.122:58714) 2024-03-16T20:45:15.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.852+0000 7fd637fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd6380b7d40 0x7fd6380b6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41770/0 (socket says 172.21.15.122:41770) 2024-03-16T20:45:15.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.852+0000 7fd63cb2c640 1 -- 172.21.15.122:0/2452667822 learned_addr learned my addr 172.21.15.122:0/2452667822 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:15.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.852+0000 7fd63d32d640 1 --2- 172.21.15.122:0/2452667822 >> v2:172.21.15.195:3300/0 conn(0x7fd638148480 0x7fd63814f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.852+0000 7fd637fff640 1 -- 172.21.15.122:0/2452667822 >> v2:172.21.15.195:3300/0 conn(0x7fd638148480 msgr2=0x7fd63814f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:15.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.852+0000 7fd637fff640 1 --2- 172.21.15.122:0/2452667822 >> v2:172.21.15.195:3300/0 conn(0x7fd638148480 0x7fd63814f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.852+0000 7fd637fff640 1 -- 172.21.15.122:0/2452667822 >> v1:172.21.15.122:6789/0 conn(0x7fd6380b7540 legacy=0x7fd63813ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.852+0000 7fd637fff640 1 -- 172.21.15.122:0/2452667822 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6380b7920 con 0x7fd6380b7d40 2024-03-16T20:45:15.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.852+0000 7fd6377fe640 1 -- 172.21.15.122:0/2452667822 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771412404 0 0) 0x7fd6380b6740 con 0x7fd6380b7540 2024-03-16T20:45:15.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd637fff640 1 --2- 172.21.15.122:0/2452667822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6380b7d40 0x7fd6380b6bd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd638071d80 tx=0x7fd62001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b703e66366084c2e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:15.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd6377fe640 1 -- 172.21.15.122:0/2452667822 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6200ac600 con 0x7fd6380b7d40 2024-03-16T20:45:15.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd6377fe640 1 -- 172.21.15.122:0/2452667822 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6200a8050 con 0x7fd6380b7d40 2024-03-16T20:45:15.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd6377fe640 1 -- 172.21.15.122:0/2452667822 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6200ac980 con 0x7fd6380b7d40 2024-03-16T20:45:15.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 -- 172.21.15.122:0/2452667822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6380b7d40 msgr2=0x7fd6380b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 --2- 172.21.15.122:0/2452667822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6380b7d40 0x7fd6380b6bd0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd638071d80 tx=0x7fd62001c3a0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 -- 172.21.15.122:0/2452667822 shutdown_connections 2024-03-16T20:45:15.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 --2- 172.21.15.122:0/2452667822 >> v2:172.21.15.195:3300/0 conn(0x7fd638148480 0x7fd63814f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 --2- 172.21.15.122:0/2452667822 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6380b7d40 0x7fd6380b6bd0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 -- 172.21.15.122:0/2452667822 >> 172.21.15.122:0/2452667822 conn(0x7fd63806fc00 msgr2=0x7fd638070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:15.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 -- 172.21.15.122:0/2452667822 shutdown_connections 2024-03-16T20:45:15.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 -- 172.21.15.122:0/2452667822 wait complete. 2024-03-16T20:45:15.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 Processor -- start 2024-03-16T20:45:15.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 -- start start 2024-03-16T20:45:15.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6380b7540 0x7fd638147e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6380b7d40 0x7fd63806c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd638148480 0x7fd638133540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6381e39a0 con 0x7fd638148480 2024-03-16T20:45:15.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd638136fe0 con 0x7fd6380b7d40 2024-03-16T20:45:15.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6381e2a30 con 0x7fd6380b7540 2024-03-16T20:45:15.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63d32d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd638148480 0x7fd638133540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63d32d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd638148480 0x7fd638133540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41778/0 (socket says 172.21.15.122:41778) 2024-03-16T20:45:15.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63d32d640 1 -- 172.21.15.122:0/3988476541 learned_addr learned my addr 172.21.15.122:0/3988476541 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:15.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63cb2c640 1 --2- 172.21.15.122:0/3988476541 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6380b7540 0x7fd638147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd637fff640 1 --2- 172.21.15.122:0/3988476541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6380b7d40 0x7fd63806c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63d32d640 1 -- 172.21.15.122:0/3988476541 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6380b7540 msgr2=0x7fd638147e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63d32d640 1 --2- 172.21.15.122:0/3988476541 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6380b7540 0x7fd638147e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63d32d640 1 -- 172.21.15.122:0/3988476541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6380b7d40 msgr2=0x7fd63806c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:15.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63d32d640 1 --2- 172.21.15.122:0/3988476541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6380b7d40 0x7fd63806c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:15.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63d32d640 1 -- 172.21.15.122:0/3988476541 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd620009420 con 0x7fd638148480 2024-03-16T20:45:15.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63cb2c640 1 --2- 172.21.15.122:0/3988476541 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6380b7540 0x7fd638147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:15.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63d32d640 1 --2- 172.21.15.122:0/3988476541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd638148480 0x7fd638133540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd628007e60 tx=0x7fd62800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:15.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd6357fa640 1 -- 172.21.15.122:0/3988476541 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628014420 con 0x7fd638148480 2024-03-16T20:45:15.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd6357fa640 1 -- 172.21.15.122:0/3988476541 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6280145a0 con 0x7fd638148480 2024-03-16T20:45:15.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 -- 172.21.15.122:0/3988476541 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd638136a20 con 0x7fd638148480 2024-03-16T20:45:15.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd6357fa640 1 -- 172.21.15.122:0/3988476541 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd628014950 con 0x7fd638148480 2024-03-16T20:45:15.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.856+0000 7fd63ed90640 1 -- 172.21.15.122:0/3988476541 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd638137760 con 0x7fd638148480 2024-03-16T20:45:15.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.860+0000 7fd63ed90640 1 -- 172.21.15.122:0/3988476541 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63806c340 con 0x7fd638148480 2024-03-16T20:45:15.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.860+0000 7fd6357fa640 1 -- 172.21.15.122:0/3988476541 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fd628014ab0 con 0x7fd638148480 2024-03-16T20:45:15.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.860+0000 7fd6357fa640 1 --2- 172.21.15.122:0/3988476541 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd618066e40 0x7fd618069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:15.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.860+0000 7fd6357fa640 1 -- 172.21.15.122:0/3988476541 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd628085390 con 0x7fd638148480 2024-03-16T20:45:15.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.860+0000 7fd63cb2c640 1 --2- 172.21.15.122:0/3988476541 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd618066e40 0x7fd618069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:15.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.860+0000 7fd63cb2c640 1 --2- 172.21.15.122:0/3988476541 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd618066e40 0x7fd618069300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd630000f70 tx=0x7fd630009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:15.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:15.864+0000 7fd6357fa640 1 -- 172.21.15.122:0/3988476541 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd63806c340 con 0x7fd638148480 2024-03-16T20:45:16.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.016+0000 7fd63ed90640 1 -- 172.21.15.122:0/3988476541 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd63806c340 con 0x7fd638148480 2024-03-16T20:45:16.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.016+0000 7fd6357fa640 1 -- 172.21.15.122:0/3988476541 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd63806c340 con 0x7fd638148480 2024-03-16T20:45:16.020 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-03-16T20:45:16.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.020+0000 7fd63ed90640 1 -- 172.21.15.122:0/3988476541 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd618066e40 msgr2=0x7fd618069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.020+0000 7fd63ed90640 1 --2- 172.21.15.122:0/3988476541 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd618066e40 0x7fd618069300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd630000f70 tx=0x7fd630009040 comp rx=0 tx=0).stop 2024-03-16T20:45:16.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.020+0000 7fd63ed90640 1 -- 172.21.15.122:0/3988476541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd638148480 msgr2=0x7fd638133540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.020+0000 7fd63ed90640 1 --2- 172.21.15.122:0/3988476541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd638148480 0x7fd638133540 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd628007e60 tx=0x7fd62800e4b0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.020+0000 7fd63ed90640 1 -- 172.21.15.122:0/3988476541 shutdown_connections 2024-03-16T20:45:16.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.020+0000 7fd63ed90640 1 --2- 172.21.15.122:0/3988476541 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd618066e40 0x7fd618069300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.020+0000 7fd63ed90640 1 --2- 172.21.15.122:0/3988476541 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd638148480 0x7fd638133540 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.020+0000 7fd63ed90640 1 --2- 172.21.15.122:0/3988476541 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6380b7d40 0x7fd63806c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.020+0000 7fd63ed90640 1 --2- 172.21.15.122:0/3988476541 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6380b7540 0x7fd638147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.020+0000 7fd63ed90640 1 -- 172.21.15.122:0/3988476541 >> 172.21.15.122:0/3988476541 conn(0x7fd63806fc00 msgr2=0x7fd6380b5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:16.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.020+0000 7fd63ed90640 1 -- 172.21.15.122:0/3988476541 shutdown_connections 2024-03-16T20:45:16.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.020+0000 7fd63ed90640 1 -- 172.21.15.122:0/3988476541 wait complete. 2024-03-16T20:45:16.038 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T20:45:16.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd60d4f1640 1 Processor -- start 2024-03-16T20:45:16.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd60d4f1640 1 -- start start 2024-03-16T20:45:16.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd60d4f1640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd6080b7480 0x7fd6080b7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd60d4f1640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd60813ed70 0x7fd60813f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd60d4f1640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd608144f10 con 0x7fd6080b3a30 2024-03-16T20:45:16.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd60d4f1640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd60813f740 con 0x7fd6080b7480 2024-03-16T20:45:16.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd60d4f1640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6080b2660 con 0x7fd60813ed70 2024-03-16T20:45:16.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd606ffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd6080b7480 0x7fd6080b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd6067fc640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fd6080b3a30 0x7fd608145940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58730/0 (socket says 172.21.15.122:58730) 2024-03-16T20:45:16.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd6067fc640 1 -- 172.21.15.122:0/406367030 learned_addr learned my addr 172.21.15.122:0/406367030 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:16.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd6077fe640 1 --2- 172.21.15.122:0/406367030 >> v2:172.21.15.195:3300/0 conn(0x7fd60813ed70 0x7fd60813f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd606ffd640 1 -- 172.21.15.122:0/406367030 >> v2:172.21.15.195:3300/0 conn(0x7fd60813ed70 msgr2=0x7fd60813f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:16.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd606ffd640 1 --2- 172.21.15.122:0/406367030 >> v2:172.21.15.195:3300/0 conn(0x7fd60813ed70 0x7fd60813f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd606ffd640 1 -- 172.21.15.122:0/406367030 >> v1:172.21.15.122:6789/0 conn(0x7fd6080b3a30 legacy=0x7fd608145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd606ffd640 1 -- 172.21.15.122:0/406367030 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6080b32c0 con 0x7fd6080b7480 2024-03-16T20:45:16.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd605ffb640 1 -- 172.21.15.122:0/406367030 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640691330 0 0) 0x7fd608144f10 con 0x7fd6080b3a30 2024-03-16T20:45:16.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd606ffd640 1 --2- 172.21.15.122:0/406367030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6080b7480 0x7fd6080b7860 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd6080ba850 tx=0x7fd5fc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13330b1eab7227a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:16.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd605ffb640 1 -- 172.21.15.122:0/406367030 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc0ab600 con 0x7fd6080b7480 2024-03-16T20:45:16.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd605ffb640 1 -- 172.21.15.122:0/406367030 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5fc0a7050 con 0x7fd6080b7480 2024-03-16T20:45:16.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.144+0000 7fd605ffb640 1 -- 172.21.15.122:0/406367030 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc0ab980 con 0x7fd6080b7480 2024-03-16T20:45:16.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 -- 172.21.15.122:0/406367030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6080b7480 msgr2=0x7fd6080b7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 --2- 172.21.15.122:0/406367030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6080b7480 0x7fd6080b7860 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd6080ba850 tx=0x7fd5fc01c3f0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 -- 172.21.15.122:0/406367030 shutdown_connections 2024-03-16T20:45:16.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 --2- 172.21.15.122:0/406367030 >> v2:172.21.15.195:3300/0 conn(0x7fd60813ed70 0x7fd60813f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 --2- 172.21.15.122:0/406367030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6080b7480 0x7fd6080b7860 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 -- 172.21.15.122:0/406367030 >> 172.21.15.122:0/406367030 conn(0x7fd60806fc90 msgr2=0x7fd6080700a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:16.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 -- 172.21.15.122:0/406367030 shutdown_connections 2024-03-16T20:45:16.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 -- 172.21.15.122:0/406367030 wait complete. 2024-03-16T20:45:16.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 Processor -- start 2024-03-16T20:45:16.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 -- start start 2024-03-16T20:45:16.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6080b3a30 0x7fd6081412c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6080b7480 0x7fd608142d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd60813ed70 0x7fd60813fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6081e09c0 con 0x7fd60813ed70 2024-03-16T20:45:16.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6081e0c60 con 0x7fd6080b7480 2024-03-16T20:45:16.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080a8660 con 0x7fd6080b3a30 2024-03-16T20:45:16.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd60813ed70 0x7fd60813fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd6077fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd60813ed70 0x7fd60813fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41792/0 (socket says 172.21.15.122:41792) 2024-03-16T20:45:16.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd6077fe640 1 -- 172.21.15.122:0/689192346 learned_addr learned my addr 172.21.15.122:0/689192346 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:16.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd606ffd640 1 --2- 172.21.15.122:0/689192346 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6080b3a30 0x7fd6081412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd6067fc640 1 --2- 172.21.15.122:0/689192346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6080b7480 0x7fd608142d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd6077fe640 1 -- 172.21.15.122:0/689192346 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6080b3a30 msgr2=0x7fd6081412c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd6077fe640 1 --2- 172.21.15.122:0/689192346 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6080b3a30 0x7fd6081412c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd6077fe640 1 -- 172.21.15.122:0/689192346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6080b7480 msgr2=0x7fd608142d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd6077fe640 1 --2- 172.21.15.122:0/689192346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6080b7480 0x7fd608142d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd6077fe640 1 -- 172.21.15.122:0/689192346 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5fc009420 con 0x7fd60813ed70 2024-03-16T20:45:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd606ffd640 1 --2- 172.21.15.122:0/689192346 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6080b3a30 0x7fd6081412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd6077fe640 1 --2- 172.21.15.122:0/689192346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd60813ed70 0x7fd60813fd90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd600007e60 tx=0x7fd60000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd5ef7fe640 1 -- 172.21.15.122:0/689192346 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd600014420 con 0x7fd60813ed70 2024-03-16T20:45:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd5ef7fe640 1 -- 172.21.15.122:0/689192346 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6000145a0 con 0x7fd60813ed70 2024-03-16T20:45:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 -- 172.21.15.122:0/689192346 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd608141800 con 0x7fd60813ed70 2024-03-16T20:45:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd5ef7fe640 1 -- 172.21.15.122:0/689192346 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd600014950 con 0x7fd60813ed70 2024-03-16T20:45:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 -- 172.21.15.122:0/689192346 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd60814f4c0 con 0x7fd60813ed70 2024-03-16T20:45:16.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.148+0000 7fd60d4f1640 1 -- 172.21.15.122:0/689192346 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6081e0e80 con 0x7fd60813ed70 2024-03-16T20:45:16.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.152+0000 7fd5ef7fe640 1 -- 172.21.15.122:0/689192346 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fd600014ab0 con 0x7fd60813ed70 2024-03-16T20:45:16.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.152+0000 7fd5ef7fe640 1 --2- 172.21.15.122:0/689192346 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd5f4066e40 0x7fd5f4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.152+0000 7fd5ef7fe640 1 -- 172.21.15.122:0/689192346 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6000852c0 con 0x7fd60813ed70 2024-03-16T20:45:16.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.152+0000 7fd606ffd640 1 --2- 172.21.15.122:0/689192346 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd5f4066e40 0x7fd5f4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.152+0000 7fd606ffd640 1 --2- 172.21.15.122:0/689192346 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd5f4066e40 0x7fd5f4069300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0ad000 tx=0x7fd5fc0a4f90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:16.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.156+0000 7fd5ef7fe640 1 -- 172.21.15.122:0/689192346 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd600052140 con 0x7fd60813ed70 2024-03-16T20:45:16.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.340+0000 7fd60d4f1640 1 -- 172.21.15.122:0/689192346 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd608075280 con 0x7fd60813ed70 2024-03-16T20:45:16.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.340+0000 7fd5ef7fe640 1 -- 172.21.15.122:0/689192346 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3397 (secure 0 0 0) 0x7fd600055800 con 0x7fd60813ed70 2024-03-16T20:45:16.345 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:16.345 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3481092793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3481092793},{"type":"v1","addr":"172.21.15.122:6833","nonce":3481092793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T20:45:04.711887+0000","modified":"2024-03-16T20:45:06.765685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/1428608180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1428608180},{"type":"v1","addr":"172.21.15.195:6835","nonce":1428608180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-03-16T20:45:16.346 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 6 2024-03-16T20:45:16.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.344+0000 7fd60d4f1640 1 -- 172.21.15.122:0/689192346 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd5f4066e40 msgr2=0x7fd5f4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.344+0000 7fd60d4f1640 1 --2- 172.21.15.122:0/689192346 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd5f4066e40 0x7fd5f4069300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc0ad000 tx=0x7fd5fc0a4f90 comp rx=0 tx=0).stop 2024-03-16T20:45:16.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.344+0000 7fd60d4f1640 1 -- 172.21.15.122:0/689192346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd60813ed70 msgr2=0x7fd60813fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.344+0000 7fd60d4f1640 1 --2- 172.21.15.122:0/689192346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd60813ed70 0x7fd60813fd90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd600007e60 tx=0x7fd60000e4b0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.344+0000 7fd60d4f1640 1 -- 172.21.15.122:0/689192346 shutdown_connections 2024-03-16T20:45:16.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.344+0000 7fd60d4f1640 1 --2- 172.21.15.122:0/689192346 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd5f4066e40 0x7fd5f4069300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.344+0000 7fd60d4f1640 1 --2- 172.21.15.122:0/689192346 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd60813ed70 0x7fd60813fd90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.344+0000 7fd60d4f1640 1 --2- 172.21.15.122:0/689192346 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6080b7480 0x7fd608142d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.344+0000 7fd60d4f1640 1 --2- 172.21.15.122:0/689192346 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6080b3a30 0x7fd6081412c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.344+0000 7fd60d4f1640 1 -- 172.21.15.122:0/689192346 >> 172.21.15.122:0/689192346 conn(0x7fd60806fc90 msgr2=0x7fd6080b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:16.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.344+0000 7fd60d4f1640 1 -- 172.21.15.122:0/689192346 shutdown_connections 2024-03-16T20:45:16.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.344+0000 7fd60d4f1640 1 -- 172.21.15.122:0/689192346 wait complete. 2024-03-16T20:45:16.363 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:16.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.472+0000 7fd1bd4e3640 1 Processor -- start 2024-03-16T20:45:16.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.472+0000 7fd1bd4e3640 1 -- start start 2024-03-16T20:45:16.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.472+0000 7fd1bd4e3640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd1b80b3830 0x7fd1b80b7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.472+0000 7fd1bd4e3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd1b80b8500 0x7fd1b80b8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.472+0000 7fd1bd4e3640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1b80b7b90 con 0x7fd1b80b77b0 2024-03-16T20:45:16.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.472+0000 7fd1bd4e3640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1b80b3500 con 0x7fd1b80b3830 2024-03-16T20:45:16.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.472+0000 7fd1bd4e3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1b813f8d0 con 0x7fd1b80b8500 2024-03-16T20:45:16.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.472+0000 7fd1affff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd1b80b3830 0x7fd1b80b7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.472+0000 7fd1affff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd1b80b3830 0x7fd1b80b7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41804/0 (socket says 172.21.15.122:41804) 2024-03-16T20:45:16.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1affff640 1 -- 172.21.15.122:0/3201092237 learned_addr learned my addr 172.21.15.122:0/3201092237 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:16.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1b77fe640 1 --2- 172.21.15.122:0/3201092237 >> v2:172.21.15.195:3300/0 conn(0x7fd1b80b8500 0x7fd1b80b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1affff640 1 -- 172.21.15.122:0/3201092237 >> v2:172.21.15.195:3300/0 conn(0x7fd1b80b8500 msgr2=0x7fd1b80b8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:16.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1affff640 1 --2- 172.21.15.122:0/3201092237 >> v2:172.21.15.195:3300/0 conn(0x7fd1b80b8500 0x7fd1b80b8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1affff640 1 -- 172.21.15.122:0/3201092237 >> v1:172.21.15.195:6789/0 conn(0x7fd1b80b77b0 legacy=0x7fd1b813f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:16.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1affff640 1 -- 172.21.15.122:0/3201092237 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b80b70c0 con 0x7fd1b80b3830 2024-03-16T20:45:16.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1affff640 1 --2- 172.21.15.122:0/3201092237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1b80b3830 0x7fd1b80b7fc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd1b81496e0 tx=0x7fd1a801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5564cd6c4443450e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:16.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1b67fc640 1 -- 172.21.15.122:0/3201092237 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80ab620 con 0x7fd1b80b3830 2024-03-16T20:45:16.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1b67fc640 1 -- 172.21.15.122:0/3201092237 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a80ab780 con 0x7fd1b80b3830 2024-03-16T20:45:16.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1b67fc640 1 -- 172.21.15.122:0/3201092237 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80abb10 con 0x7fd1b80b3830 2024-03-16T20:45:16.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1bd4e3640 1 -- 172.21.15.122:0/3201092237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1b80b3830 msgr2=0x7fd1b80b7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1bd4e3640 1 --2- 172.21.15.122:0/3201092237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1b80b3830 0x7fd1b80b7fc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd1b81496e0 tx=0x7fd1a801c420 comp rx=0 tx=0).stop 2024-03-16T20:45:16.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1bd4e3640 1 -- 172.21.15.122:0/3201092237 shutdown_connections 2024-03-16T20:45:16.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1bd4e3640 1 --2- 172.21.15.122:0/3201092237 >> v2:172.21.15.195:3300/0 conn(0x7fd1b80b8500 0x7fd1b80b8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1bd4e3640 1 --2- 172.21.15.122:0/3201092237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1b80b3830 0x7fd1b80b7fc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1bd4e3640 1 -- 172.21.15.122:0/3201092237 >> 172.21.15.122:0/3201092237 conn(0x7fd1b8070d30 msgr2=0x7fd1b8071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:16.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1bd4e3640 1 -- 172.21.15.122:0/3201092237 shutdown_connections 2024-03-16T20:45:16.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.476+0000 7fd1bd4e3640 1 -- 172.21.15.122:0/3201092237 wait complete. 2024-03-16T20:45:16.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1bd4e3640 1 Processor -- start 2024-03-16T20:45:16.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1bd4e3640 1 -- start start 2024-03-16T20:45:16.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1bd4e3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1b80b3830 0x7fd1b813e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1bd4e3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1b80b77b0 0x7fd1b81df3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1bd4e3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1b80b8500 0x7fd1b8144f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1bd4e3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b813ea80 con 0x7fd1b80b77b0 2024-03-16T20:45:16.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1bd4e3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b813ec70 con 0x7fd1b80b8500 2024-03-16T20:45:16.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1bd4e3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b80b25c0 con 0x7fd1b80b3830 2024-03-16T20:45:16.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1affff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1b80b77b0 0x7fd1b81df3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1affff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1b80b77b0 0x7fd1b81df3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41810/0 (socket says 172.21.15.122:41810) 2024-03-16T20:45:16.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1affff640 1 -- 172.21.15.122:0/1675734824 learned_addr learned my addr 172.21.15.122:0/1675734824 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:16.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1b6ffd640 1 --2- 172.21.15.122:0/1675734824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1b80b3830 0x7fd1b813e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1b77fe640 1 --2- 172.21.15.122:0/1675734824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1b80b8500 0x7fd1b8144f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1affff640 1 -- 172.21.15.122:0/1675734824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1b80b3830 msgr2=0x7fd1b813e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1affff640 1 --2- 172.21.15.122:0/1675734824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1b80b3830 0x7fd1b813e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1affff640 1 -- 172.21.15.122:0/1675734824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1b80b8500 msgr2=0x7fd1b8144f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1affff640 1 --2- 172.21.15.122:0/1675734824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1b80b8500 0x7fd1b8144f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1affff640 1 -- 172.21.15.122:0/1675734824 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a80094d0 con 0x7fd1b80b77b0 2024-03-16T20:45:16.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1b77fe640 1 --2- 172.21.15.122:0/1675734824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1b80b8500 0x7fd1b8144f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:16.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1b6ffd640 1 --2- 172.21.15.122:0/1675734824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1b80b3830 0x7fd1b813e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:16.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1affff640 1 --2- 172.21.15.122:0/1675734824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1b80b77b0 0x7fd1b81df3a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd1a801a050 tx=0x7fd1a801ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:16.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1af7fe640 1 -- 172.21.15.122:0/1675734824 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80abe80 con 0x7fd1b80b77b0 2024-03-16T20:45:16.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1af7fe640 1 -- 172.21.15.122:0/1675734824 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a80be3f0 con 0x7fd1b80b77b0 2024-03-16T20:45:16.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1bd4e3640 1 -- 172.21.15.122:0/1675734824 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b81e0340 con 0x7fd1b80b77b0 2024-03-16T20:45:16.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1af7fe640 1 -- 172.21.15.122:0/1675734824 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a80be780 con 0x7fd1b80b77b0 2024-03-16T20:45:16.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1bd4e3640 1 -- 172.21.15.122:0/1675734824 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b80b2250 con 0x7fd1b80b77b0 2024-03-16T20:45:16.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1bd4e3640 1 -- 172.21.15.122:0/1675734824 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b80b2750 con 0x7fd1b80b77b0 2024-03-16T20:45:16.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1af7fe640 1 -- 172.21.15.122:0/1675734824 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fd1a80c7440 con 0x7fd1b80b77b0 2024-03-16T20:45:16.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1af7fe640 1 --2- 172.21.15.122:0/1675734824 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd198066e40 0x7fd198069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1af7fe640 1 -- 172.21.15.122:0/1675734824 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1a811f270 con 0x7fd1b80b77b0 2024-03-16T20:45:16.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.480+0000 7fd1b6ffd640 1 --2- 172.21.15.122:0/1675734824 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd198066e40 0x7fd198069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.484+0000 7fd1b6ffd640 1 --2- 172.21.15.122:0/1675734824 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd198066e40 0x7fd198069300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0003ec0 tx=0x7fd1a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:16.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.484+0000 7fd1af7fe640 1 -- 172.21.15.122:0/1675734824 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd1a80ec0f0 con 0x7fd1b80b77b0 2024-03-16T20:45:16.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.632+0000 7fd1bd4e3640 1 -- 172.21.15.122:0/1675734824 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1b8075280 con 0x7fd1b80b77b0 2024-03-16T20:45:16.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.636+0000 7fd1af7fe640 1 -- 172.21.15.122:0/1675734824 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7fd1a80ef7b0 con 0x7fd1b80b77b0 2024-03-16T20:45:16.639 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:16.639 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:06.404479+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:16.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.636+0000 7fd1bd4e3640 1 -- 172.21.15.122:0/1675734824 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd198066e40 msgr2=0x7fd198069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.636+0000 7fd1bd4e3640 1 --2- 172.21.15.122:0/1675734824 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd198066e40 0x7fd198069300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0003ec0 tx=0x7fd1a0009040 comp rx=0 tx=0).stop 2024-03-16T20:45:16.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.636+0000 7fd1bd4e3640 1 -- 172.21.15.122:0/1675734824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1b80b77b0 msgr2=0x7fd1b81df3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.636+0000 7fd1bd4e3640 1 --2- 172.21.15.122:0/1675734824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1b80b77b0 0x7fd1b81df3a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd1a801a050 tx=0x7fd1a801ac60 comp rx=0 tx=0).stop 2024-03-16T20:45:16.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.636+0000 7fd1bd4e3640 1 -- 172.21.15.122:0/1675734824 shutdown_connections 2024-03-16T20:45:16.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.636+0000 7fd1bd4e3640 1 --2- 172.21.15.122:0/1675734824 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd198066e40 0x7fd198069300 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.636+0000 7fd1bd4e3640 1 --2- 172.21.15.122:0/1675734824 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd1b80b8500 0x7fd1b8144f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.636+0000 7fd1bd4e3640 1 --2- 172.21.15.122:0/1675734824 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd1b80b77b0 0x7fd1b81df3a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.636+0000 7fd1bd4e3640 1 --2- 172.21.15.122:0/1675734824 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd1b80b3830 0x7fd1b813e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.636+0000 7fd1bd4e3640 1 -- 172.21.15.122:0/1675734824 >> 172.21.15.122:0/1675734824 conn(0x7fd1b8070d30 msgr2=0x7fd1b813c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:16.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.636+0000 7fd1bd4e3640 1 -- 172.21.15.122:0/1675734824 shutdown_connections 2024-03-16T20:45:16.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.636+0000 7fd1bd4e3640 1 -- 172.21.15.122:0/1675734824 wait complete. 2024-03-16T20:45:16.656 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T20:45:16.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.756+0000 7f08865e8640 1 Processor -- start 2024-03-16T20:45:16.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.756+0000 7f08865e8640 1 -- start start 2024-03-16T20:45:16.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f08865e8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f088013d8d0 0x7f088006b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f08865e8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0880146a50 0x7f0880143460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f08865e8640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08800b4c50 con 0x7f0880071580 2024-03-16T20:45:16.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f08865e8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08800b2c30 con 0x7f088013d8d0 2024-03-16T20:45:16.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f08865e8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0880146ee0 con 0x7f0880146a50 2024-03-16T20:45:16.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f0884b85640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f0880071580 0x7f08800b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:58734/0 (socket says 172.21.15.122:58734) 2024-03-16T20:45:16.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f0884b85640 1 -- 172.21.15.122:0/71388880 learned_addr learned my addr 172.21.15.122:0/71388880 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:16.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087ffff640 1 --2- 172.21.15.122:0/71388880 >> v2:172.21.15.195:3300/0 conn(0x7f088013d8d0 0x7f088006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087f7fe640 1 --2- 172.21.15.122:0/71388880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0880146a50 0x7f0880143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 -- 172.21.15.122:0/71388880 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602753032 0 0) 0x7f08800b4c50 con 0x7f0880071580 2024-03-16T20:45:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 -- 172.21.15.122:0/71388880 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f086c004640 con 0x7f0880071580 2024-03-16T20:45:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 -- 172.21.15.122:0/71388880 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1287030203 0 0) 0x7f086c004640 con 0x7f0880071580 2024-03-16T20:45:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 -- 172.21.15.122:0/71388880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0880146a50 msgr2=0x7f0880143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 --2- 172.21.15.122:0/71388880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0880146a50 0x7f0880143460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 -- 172.21.15.122:0/71388880 >> v2:172.21.15.195:3300/0 conn(0x7f088013d8d0 msgr2=0x7f088006b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 --2- 172.21.15.122:0/71388880 >> v2:172.21.15.195:3300/0 conn(0x7f088013d8d0 0x7f088006b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 -- 172.21.15.122:0/71388880 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08801471c0 con 0x7f0880071580 2024-03-16T20:45:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 -- 172.21.15.122:0/71388880 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f0870001680 con 0x7f0880071580 2024-03-16T20:45:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 -- 172.21.15.122:0/71388880 >> v1:172.21.15.122:6789/0 conn(0x7f0880071580 legacy=0x7f08800b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 --2- 172.21.15.122:0/71388880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f086c004a90 0x7f086c004eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 --2- 172.21.15.122:0/71388880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f086c0058e0 0x7f086c005d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 --2- 172.21.15.122:0/71388880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f086c0062a0 0x7f086c00caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 -- 172.21.15.122:0/71388880 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0880146ee0 con 0x7f086c004a90 2024-03-16T20:45:16.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 -- 172.21.15.122:0/71388880 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f086c005600 con 0x7f086c0058e0 2024-03-16T20:45:16.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 -- 172.21.15.122:0/71388880 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086c00cfe0 con 0x7f086c0062a0 2024-03-16T20:45:16.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087ffff640 1 --2- 172.21.15.122:0/71388880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f086c004a90 0x7f086c004eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087f7fe640 1 --2- 172.21.15.122:0/71388880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f086c0058e0 0x7f086c005d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087ffff640 1 -- 172.21.15.122:0/71388880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f086c0062a0 msgr2=0x7f086c00caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:16.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087ffff640 1 --2- 172.21.15.122:0/71388880 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f086c0062a0 0x7f086c00caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087ffff640 1 -- 172.21.15.122:0/71388880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f086c0058e0 msgr2=0x7f086c005d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:16.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087ffff640 1 --2- 172.21.15.122:0/71388880 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f086c0058e0 0x7f086c005d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087ffff640 1 -- 172.21.15.122:0/71388880 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086c004640 con 0x7f086c004a90 2024-03-16T20:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087ffff640 1 --2- 172.21.15.122:0/71388880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f086c004a90 0x7f086c004eb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0878009fc0 tx=0x7f087801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7feb3dcfdaa31995 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 -- 172.21.15.122:0/71388880 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08780a5b40 con 0x7f086c004a90 2024-03-16T20:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 -- 172.21.15.122:0/71388880 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087800f040 con 0x7f086c004a90 2024-03-16T20:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.760+0000 7f087effd640 1 -- 172.21.15.122:0/71388880 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08780ad440 con 0x7f086c004a90 2024-03-16T20:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f08865e8640 1 -- 172.21.15.122:0/71388880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f086c004a90 msgr2=0x7f086c004eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f08865e8640 1 --2- 172.21.15.122:0/71388880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f086c004a90 0x7f086c004eb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0878009fc0 tx=0x7f087801ccc0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f0884b85640 1 -- 172.21.15.122:0/71388880 reap_dead start 2024-03-16T20:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f08865e8640 1 -- 172.21.15.122:0/71388880 shutdown_connections 2024-03-16T20:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f08865e8640 1 -- 172.21.15.122:0/71388880 >> 172.21.15.122:0/71388880 conn(0x7f08800b8860 msgr2=0x7f08800b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f08865e8640 1 -- 172.21.15.122:0/71388880 shutdown_connections 2024-03-16T20:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f08865e8640 1 -- 172.21.15.122:0/71388880 wait complete. 2024-03-16T20:45:16.769 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f08865e8640 1 Processor -- start 2024-03-16T20:45:16.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f08865e8640 1 -- start start 2024-03-16T20:45:16.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f08865e8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f086c004a90 0x7f088013e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f08865e8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0880143880 0x7f0880143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f08865e8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08801dce40 0x7f08801dd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f08865e8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088013df10 con 0x7f08801dce40 2024-03-16T20:45:16.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f08865e8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088013e1f0 con 0x7f0880143880 2024-03-16T20:45:16.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f08865e8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088013e4d0 con 0x7f086c004a90 2024-03-16T20:45:16.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f0884b85640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08801dce40 0x7f08801dd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f0884b85640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08801dce40 0x7f08801dd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41838/0 (socket says 172.21.15.122:41838) 2024-03-16T20:45:16.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f0884b85640 1 -- 172.21.15.122:0/3965933336 learned_addr learned my addr 172.21.15.122:0/3965933336 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:16.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f087f7fe640 1 --2- 172.21.15.122:0/3965933336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0880143880 0x7f0880143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f087ffff640 1 --2- 172.21.15.122:0/3965933336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f086c004a90 0x7f088013e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f0884b85640 1 -- 172.21.15.122:0/3965933336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f086c004a90 msgr2=0x7f088013e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.770 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f0884b85640 1 --2- 172.21.15.122:0/3965933336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f086c004a90 0x7f088013e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f0884b85640 1 -- 172.21.15.122:0/3965933336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0880143880 msgr2=0x7f0880143d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f0884b85640 1 --2- 172.21.15.122:0/3965933336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0880143880 0x7f0880143d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f0884b85640 1 -- 172.21.15.122:0/3965933336 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088013ef30 con 0x7f08801dce40 2024-03-16T20:45:16.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f0884b85640 1 --2- 172.21.15.122:0/3965933336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08801dce40 0x7f08801dd2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0870001060 tx=0x7f087000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:16.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f087cff9640 1 -- 172.21.15.122:0/3965933336 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08700072d0 con 0x7f08801dce40 2024-03-16T20:45:16.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f087cff9640 1 -- 172.21.15.122:0/3965933336 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0870007430 con 0x7f08801dce40 2024-03-16T20:45:16.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f08865e8640 1 -- 172.21.15.122:0/3965933336 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088013f4d0 con 0x7f08801dce40 2024-03-16T20:45:16.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f087cff9640 1 -- 172.21.15.122:0/3965933336 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087000ce50 con 0x7f08801dce40 2024-03-16T20:45:16.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f08865e8640 1 -- 172.21.15.122:0/3965933336 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0880144620 con 0x7f08801dce40 2024-03-16T20:45:16.771 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.764+0000 7f08865e8640 1 -- 172.21.15.122:0/3965933336 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088013dd70 con 0x7f08801dce40 2024-03-16T20:45:16.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.768+0000 7f087cff9640 1 -- 172.21.15.122:0/3965933336 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f08700047f0 con 0x7f08801dce40 2024-03-16T20:45:16.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.768+0000 7f087cff9640 1 --2- 172.21.15.122:0/3965933336 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0860066e70 0x7f0860069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:16.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.768+0000 7f087cff9640 1 -- 172.21.15.122:0/3965933336 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0870084390 con 0x7f08801dce40 2024-03-16T20:45:16.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.768+0000 7f087ffff640 1 --2- 172.21.15.122:0/3965933336 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0860066e70 0x7f0860069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:16.772 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.768+0000 7f087ffff640 1 --2- 172.21.15.122:0/3965933336 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0860066e70 0x7f0860069330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f08780b0040 tx=0x7f08780a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:16.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.772+0000 7f087cff9640 1 -- 172.21.15.122:0/3965933336 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f088013dd70 con 0x7f08801dce40 2024-03-16T20:45:16.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.948+0000 7f08865e8640 1 -- 172.21.15.122:0/3965933336 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f088013dd70 con 0x7f08801dce40 2024-03-16T20:45:16.951 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.948+0000 7f087cff9640 1 -- 172.21.15.122:0/3965933336 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3397 (secure 0 0 0) 0x7f088013dd70 con 0x7f08801dce40 2024-03-16T20:45:16.952 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:16.952 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3481092793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3481092793},{"type":"v1","addr":"172.21.15.122:6833","nonce":3481092793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T20:45:04.711887+0000","modified":"2024-03-16T20:45:06.765685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/1428608180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1428608180},{"type":"v1","addr":"172.21.15.195:6835","nonce":1428608180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-03-16T20:45:16.952 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 6 2024-03-16T20:45:16.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.948+0000 7f08865e8640 1 -- 172.21.15.122:0/3965933336 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0860066e70 msgr2=0x7f0860069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.948+0000 7f08865e8640 1 --2- 172.21.15.122:0/3965933336 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0860066e70 0x7f0860069330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f08780b0040 tx=0x7f08780a7040 comp rx=0 tx=0).stop 2024-03-16T20:45:16.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.948+0000 7f08865e8640 1 -- 172.21.15.122:0/3965933336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08801dce40 msgr2=0x7f08801dd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:16.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.948+0000 7f08865e8640 1 --2- 172.21.15.122:0/3965933336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08801dce40 0x7f08801dd2f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f0870001060 tx=0x7f087000cb10 comp rx=0 tx=0).stop 2024-03-16T20:45:16.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.952+0000 7f08865e8640 1 -- 172.21.15.122:0/3965933336 shutdown_connections 2024-03-16T20:45:16.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.952+0000 7f08865e8640 1 --2- 172.21.15.122:0/3965933336 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0860066e70 0x7f0860069330 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.952+0000 7f08865e8640 1 --2- 172.21.15.122:0/3965933336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08801dce40 0x7f08801dd2f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.952+0000 7f08865e8640 1 --2- 172.21.15.122:0/3965933336 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0880143880 0x7f0880143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.952+0000 7f08865e8640 1 --2- 172.21.15.122:0/3965933336 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f086c004a90 0x7f088013e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:16.954 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.952+0000 7f08865e8640 1 -- 172.21.15.122:0/3965933336 >> 172.21.15.122:0/3965933336 conn(0x7f08800b8860 msgr2=0x7f08800b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:16.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.952+0000 7f08865e8640 1 -- 172.21.15.122:0/3965933336 shutdown_connections 2024-03-16T20:45:16.955 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:16.952+0000 7f08865e8640 1 -- 172.21.15.122:0/3965933336 wait complete. 2024-03-16T20:45:17.170 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-16T20:45:04.711887+0000', 'modified': '2024-03-16T20:45:06.765685+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4251}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4251': {'gid': 4251, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.195:6835/1428608180', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.195:6834', 'nonce': 1428608180}, {'type': 'v1', 'addr': '172.21.15.195:6835', 'nonce': 1428608180}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4251, 'qdb_cluster': [4251]} 2024-03-16T20:45:17.171 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-16T20:45:17.171 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-16T20:45:17.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9b1a4e640 1 Processor -- start 2024-03-16T20:45:17.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9b1a4e640 1 -- start start 2024-03-16T20:45:17.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9b1a4e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff9ac06b1f0 0x7ff9ac0b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9b1a4e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9ac06f6f0 0x7ff9ac141270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9b1a4e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff9ac1417b0 0x7ff9ac14a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9b1a4e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ac147ca0 con 0x7ff9ac06b1f0 2024-03-16T20:45:17.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9b1a4e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ac06b5d0 con 0x7ff9ac1417b0 2024-03-16T20:45:17.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9b1a4e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac147650 con 0x7ff9ac06f6f0 2024-03-16T20:45:17.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9aaffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff9ac06b1f0 0x7ff9ac0b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9aaffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff9ac06b1f0 0x7ff9ac0b7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41854/0 (socket says 172.21.15.122:41854) 2024-03-16T20:45:17.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9aaffd640 1 -- 172.21.15.122:0/630000063 learned_addr learned my addr 172.21.15.122:0/630000063 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:17.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9aa7fc640 1 --2- 172.21.15.122:0/630000063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9ac06f6f0 0x7ff9ac141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9ab7fe640 1 --2- 172.21.15.122:0/630000063 >> v2:172.21.15.195:3300/0 conn(0x7ff9ac1417b0 0x7ff9ac14a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9aaffd640 1 -- 172.21.15.122:0/630000063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9ac06f6f0 msgr2=0x7ff9ac141270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:17.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9aaffd640 1 --2- 172.21.15.122:0/630000063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9ac06f6f0 0x7ff9ac141270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9aaffd640 1 -- 172.21.15.122:0/630000063 >> v2:172.21.15.195:3300/0 conn(0x7ff9ac1417b0 msgr2=0x7ff9ac14a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:17.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9aaffd640 1 --2- 172.21.15.122:0/630000063 >> v2:172.21.15.195:3300/0 conn(0x7ff9ac1417b0 0x7ff9ac14a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.292+0000 7ff9aaffd640 1 -- 172.21.15.122:0/630000063 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac147930 con 0x7ff9ac06b1f0 2024-03-16T20:45:17.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.296+0000 7ff9aaffd640 1 --2- 172.21.15.122:0/630000063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9ac06b1f0 0x7ff9ac0b7550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac144f30 tx=0x7ff99c01c200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d4006a0181c770 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:17.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.296+0000 7ff9a9ffb640 1 -- 172.21.15.122:0/630000063 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0ab600 con 0x7ff9ac06b1f0 2024-03-16T20:45:17.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.296+0000 7ff9a9ffb640 1 -- 172.21.15.122:0/630000063 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c0a7050 con 0x7ff9ac06b1f0 2024-03-16T20:45:17.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.296+0000 7ff9a9ffb640 1 -- 172.21.15.122:0/630000063 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0ab980 con 0x7ff9ac06b1f0 2024-03-16T20:45:17.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.296+0000 7ff9b1a4e640 1 -- 172.21.15.122:0/630000063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9ac06b1f0 msgr2=0x7ff9ac0b7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:17.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.296+0000 7ff9b1a4e640 1 --2- 172.21.15.122:0/630000063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9ac06b1f0 0x7ff9ac0b7550 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac144f30 tx=0x7ff99c01c200 comp rx=0 tx=0).stop 2024-03-16T20:45:17.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.296+0000 7ff9b1a4e640 1 -- 172.21.15.122:0/630000063 shutdown_connections 2024-03-16T20:45:17.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.296+0000 7ff9b1a4e640 1 --2- 172.21.15.122:0/630000063 >> v2:172.21.15.195:3300/0 conn(0x7ff9ac1417b0 0x7ff9ac14a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.296+0000 7ff9b1a4e640 1 --2- 172.21.15.122:0/630000063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9ac06f6f0 0x7ff9ac141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.296+0000 7ff9b1a4e640 1 --2- 172.21.15.122:0/630000063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9ac06b1f0 0x7ff9ac0b7550 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.296+0000 7ff9b1a4e640 1 -- 172.21.15.122:0/630000063 >> 172.21.15.122:0/630000063 conn(0x7ff9ac070d30 msgr2=0x7ff9ac071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:17.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.296+0000 7ff9b1a4e640 1 -- 172.21.15.122:0/630000063 shutdown_connections 2024-03-16T20:45:17.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.296+0000 7ff9b1a4e640 1 -- 172.21.15.122:0/630000063 wait complete. 2024-03-16T20:45:17.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.296+0000 7ff9b1a4e640 1 Processor -- start 2024-03-16T20:45:17.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.296+0000 7ff9b1a4e640 1 -- start start 2024-03-16T20:45:17.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9b1a4e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9ac06b1f0 0x7ff9ac1442f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9b1a4e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9ac06f6f0 0x7ff9ac1e3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9b1a4e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9ac1417b0 0x7ff9ac150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9b1a4e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac1436b0 con 0x7ff9ac06b1f0 2024-03-16T20:45:17.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9b1a4e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac143ad0 con 0x7ff9ac06f6f0 2024-03-16T20:45:17.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9b1a4e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac14c400 con 0x7ff9ac1417b0 2024-03-16T20:45:17.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9aaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9ac06b1f0 0x7ff9ac1442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9aaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9ac06b1f0 0x7ff9ac1442f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41866/0 (socket says 172.21.15.122:41866) 2024-03-16T20:45:17.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9aaffd640 1 -- 172.21.15.122:0/1638824327 learned_addr learned my addr 172.21.15.122:0/1638824327 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:17.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9aa7fc640 1 --2- 172.21.15.122:0/1638824327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9ac06f6f0 0x7ff9ac1e3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9ab7fe640 1 --2- 172.21.15.122:0/1638824327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9ac1417b0 0x7ff9ac150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9aaffd640 1 -- 172.21.15.122:0/1638824327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9ac1417b0 msgr2=0x7ff9ac150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:17.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9aaffd640 1 --2- 172.21.15.122:0/1638824327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9ac1417b0 0x7ff9ac150960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9aaffd640 1 -- 172.21.15.122:0/1638824327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9ac06f6f0 msgr2=0x7ff9ac1e3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:17.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9aaffd640 1 --2- 172.21.15.122:0/1638824327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9ac06f6f0 0x7ff9ac1e3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9aaffd640 1 -- 172.21.15.122:0/1638824327 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff99c0094d0 con 0x7ff9ac06b1f0 2024-03-16T20:45:17.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9aa7fc640 1 --2- 172.21.15.122:0/1638824327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9ac06f6f0 0x7ff9ac1e3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:17.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9aaffd640 1 --2- 172.21.15.122:0/1638824327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9ac06b1f0 0x7ff9ac1442f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0ae040 tx=0x7ff99c01aaa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:17.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff993fff640 1 -- 172.21.15.122:0/1638824327 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0b6620 con 0x7ff9ac06b1f0 2024-03-16T20:45:17.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff993fff640 1 -- 172.21.15.122:0/1638824327 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c0abe90 con 0x7ff9ac06b1f0 2024-03-16T20:45:17.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9b1a4e640 1 -- 172.21.15.122:0/1638824327 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ac1518c0 con 0x7ff9ac06b1f0 2024-03-16T20:45:17.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff993fff640 1 -- 172.21.15.122:0/1638824327 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0b69e0 con 0x7ff9ac06b1f0 2024-03-16T20:45:17.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9b1a4e640 1 -- 172.21.15.122:0/1638824327 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ac143e30 con 0x7ff9ac06b1f0 2024-03-16T20:45:17.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff993fff640 1 -- 172.21.15.122:0/1638824327 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7ff99c0b2040 con 0x7ff9ac06b1f0 2024-03-16T20:45:17.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff993fff640 1 --2- 172.21.15.122:0/1638824327 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff998066e40 0x7ff998069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.313 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff993fff640 1 -- 172.21.15.122:0/1638824327 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff99c11ee90 con 0x7ff9ac06b1f0 2024-03-16T20:45:17.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9aa7fc640 1 --2- 172.21.15.122:0/1638824327 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff998066e40 0x7ff998069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9b4bc0000 1 Processor -- start 2024-03-16T20:45:17.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.300+0000 7ff9b4bc0000 1 -- start start 2024-03-16T20:45:17.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9aa7fc640 1 --2- 172.21.15.122:0/1638824327 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff998066e40 0x7ff998069300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac141d60 tx=0x7ff9a0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:17.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44ccdfdd0 0x55a44cce01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44cce06f0 0x55a44cce0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44cce1010 0x55a44cd1adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44cba7d00 con 0x55a44cce1010 2024-03-16T20:45:17.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44cb39430 con 0x55a44cce06f0 2024-03-16T20:45:17.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a44cb36960 con 0x55a44ccdfdd0 2024-03-16T20:45:17.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9ab7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44cce1010 0x55a44cd1adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9ab7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44cce1010 0x55a44cd1adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41870/0 (socket says 172.21.15.122:41870) 2024-03-16T20:45:17.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9ab7fe640 1 -- 172.21.15.122:0/2724859825 learned_addr learned my addr 172.21.15.122:0/2724859825 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:17.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9aa7fc640 1 --2- 172.21.15.122:0/2724859825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44cce06f0 0x55a44cce0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9aaffd640 1 --2- 172.21.15.122:0/2724859825 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44ccdfdd0 0x55a44cce01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9ab7fe640 1 -- 172.21.15.122:0/2724859825 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44ccdfdd0 msgr2=0x55a44cce01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:17.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9ab7fe640 1 --2- 172.21.15.122:0/2724859825 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44ccdfdd0 0x55a44cce01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9ab7fe640 1 -- 172.21.15.122:0/2724859825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44cce06f0 msgr2=0x55a44cce0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:17.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9ab7fe640 1 --2- 172.21.15.122:0/2724859825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44cce06f0 0x55a44cce0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9ab7fe640 1 -- 172.21.15.122:0/2724859825 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a44c828890 con 0x55a44cce1010 2024-03-16T20:45:17.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9aaffd640 1 --2- 172.21.15.122:0/2724859825 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44ccdfdd0 0x55a44cce01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:17.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9ab7fe640 1 --2- 172.21.15.122:0/2724859825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44cce1010 0x55a44cd1adc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009ec0 tx=0x7ff9a4007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43dacc1513af0945 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:17.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9917fa640 1 -- 172.21.15.122:0/2724859825 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4013780 con 0x55a44cce1010 2024-03-16T20:45:17.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9917fa640 1 -- 172.21.15.122:0/2724859825 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a400f050 con 0x55a44cce1010 2024-03-16T20:45:17.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9917fa640 1 -- 172.21.15.122:0/2724859825 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4013b20 con 0x55a44cce1010 2024-03-16T20:45:17.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/2724859825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44cce1010 msgr2=0x55a44cd1adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:17.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/2724859825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44cce1010 0x55a44cd1adc0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff9a4009ec0 tx=0x7ff9a4007700 comp rx=0 tx=0).stop 2024-03-16T20:45:17.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/2724859825 shutdown_connections 2024-03-16T20:45:17.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/2724859825 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44cce1010 0x55a44cd1adc0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/2724859825 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44cce06f0 0x55a44cce0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/2724859825 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44ccdfdd0 0x55a44cce01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/2724859825 >> 172.21.15.122:0/2724859825 conn(0x55a44ccdc2c0 msgr2=0x55a44ccde6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:17.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/2724859825 shutdown_connections 2024-03-16T20:45:17.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/2724859825 wait complete. 2024-03-16T20:45:17.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 -- start start 2024-03-16T20:45:17.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 Processor -- start 2024-03-16T20:45:17.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44ccdfdd0 0x55a44cd1ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44cce06f0 0x55a44cd1bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44cce1010 0x55a44ccebb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44cb32140 con 0x55a44cce06f0 2024-03-16T20:45:17.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44cacc630 con 0x55a44ccdfdd0 2024-03-16T20:45:17.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a44cacc2d0 con 0x55a44cce1010 2024-03-16T20:45:17.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 10 client.0 _ll_register_callbacks cb 0x55a44cbe21c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T20:45:17.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9aa7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44cce06f0 0x55a44cd1bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9aa7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44cce06f0 0x55a44cd1bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41876/0 (socket says 172.21.15.122:41876) 2024-03-16T20:45:17.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9aa7fc640 1 -- 172.21.15.122:0/1538005457 learned_addr learned my addr 172.21.15.122:0/1538005457 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:17.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9aaffd640 1 --2- 172.21.15.122:0/1538005457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44ccdfdd0 0x55a44cd1ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9ab7fe640 1 --2- 172.21.15.122:0/1538005457 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44cce1010 0x55a44ccebb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9aa7fc640 1 -- 172.21.15.122:0/1538005457 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44cce1010 msgr2=0x55a44ccebb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:17.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9aa7fc640 1 --2- 172.21.15.122:0/1538005457 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44cce1010 0x55a44ccebb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9aa7fc640 1 -- 172.21.15.122:0/1538005457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44ccdfdd0 msgr2=0x55a44cd1ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:17.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9aa7fc640 1 --2- 172.21.15.122:0/1538005457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44ccdfdd0 0x55a44cd1ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9aa7fc640 1 -- 172.21.15.122:0/1538005457 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a4009b20 con 0x55a44cce06f0 2024-03-16T20:45:17.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9ab7fe640 1 --2- 172.21.15.122:0/1538005457 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44cce1010 0x55a44ccebb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:17.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9aa7fc640 1 --2- 172.21.15.122:0/1538005457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44cce06f0 0x55a44cd1bfc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000ea70 tx=0x7ff9a00046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e752bbb637351c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:17.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff98a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-16T20:45:17.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff98a7fc640 1 -- 172.21.15.122:0/1538005457 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0013430 con 0x55a44cce06f0 2024-03-16T20:45:17.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff98a7fc640 1 -- 172.21.15.122:0/1538005457 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9a0013590 con 0x55a44cce06f0 2024-03-16T20:45:17.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.304+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1538005457 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a44cae5850 con 0x55a44cce06f0 2024-03-16T20:45:17.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff98a7fc640 1 -- 172.21.15.122:0/1538005457 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9a0013750 con 0x55a44cce06f0 2024-03-16T20:45:17.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 10 client.4653 fetch_fsmap learned FSMap version 6 2024-03-16T20:45:17.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1538005457 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55a44caf1e70 con 0x55a44cce06f0 2024-03-16T20:45:17.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff98a7fc640 1 -- 172.21.15.122:0/1538005457 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a44cae5850 con 0x55a44cce06f0 2024-03-16T20:45:17.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff98a7fc640 1 -- 172.21.15.122:0/1538005457 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 6) v1 ==== 2164+0+0 (secure 0 0 0) 0x7ff9a00059a0 con 0x55a44cce06f0 2024-03-16T20:45:17.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 10 client.4653 fetch_fsmap finished waiting for FSMap version 6 2024-03-16T20:45:17.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 10 client.4653 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T20:45:17.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 20 client.4653 populate_metadata read hostname 'smithi122' 2024-03-16T20:45:17.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/1538005457 >> [v2:172.21.15.195:6834/1428608180,v1:172.21.15.195:6835/1428608180] conn(0x55a44ccec820 0x55a44ccecc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 4 client.4653 mds_command: new command op to 4251 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T20:45:17.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1538005457 --> [v2:172.21.15.195:6834/1428608180,v1:172.21.15.195:6835/1428608180] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a44cb2ef20 con 0x55a44ccec820 2024-03-16T20:45:17.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9aaffd640 1 --2- 172.21.15.122:0/1538005457 >> [v2:172.21.15.195:6834/1428608180,v1:172.21.15.195:6835/1428608180] conn(0x55a44ccec820 0x55a44ccecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9aaffd640 1 --2- 172.21.15.122:0/1538005457 >> [v2:172.21.15.195:6834/1428608180,v1:172.21.15.195:6835/1428608180] conn(0x55a44ccec820 0x55a44ccecc00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a332a28115cdb0e1 server_cookie=298df1d5a5c8d623 in_seq=0 out_seq=0 2024-03-16T20:45:17.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff98a7fc640 10 client.4653 ms_handle_connect on v2:172.21.15.195:6834/1428608180 2024-03-16T20:45:17.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff98a7fc640 1 -- 172.21.15.122:0/1538005457 <== mds.0 v2:172.21.15.195:6834/1428608180 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55a44cb2ef20 con 0x55a44ccec820 2024-03-16T20:45:17.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff98a7fc640 10 client.4653 handle_command_reply: tid=0 2024-03-16T20:45:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 1 client.4653 shutdown 2024-03-16T20:45:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1538005457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44cce06f0 msgr2=0x55a44cd1bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/1538005457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44cce06f0 0x55a44cd1bfc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000ea70 tx=0x7ff9a00046d0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1538005457 shutdown_connections 2024-03-16T20:45:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/1538005457 >> [v2:172.21.15.195:6834/1428608180,v1:172.21.15.195:6835/1428608180] conn(0x55a44ccec820 0x55a44ccecc00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/1538005457 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44cce1010 0x55a44ccebb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff98a7fc640 0 client.4653 ms_handle_reset on v2:172.21.15.195:6834/1428608180 2024-03-16T20:45:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/1538005457 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44cce06f0 0x55a44cd1bfc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/1538005457 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44ccdfdd0 0x55a44cd1ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1538005457 >> 172.21.15.122:0/1538005457 conn(0x55a44ccdc2c0 msgr2=0x55a44ccdef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1538005457 shutdown_connections 2024-03-16T20:45:17.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1538005457 wait complete. 2024-03-16T20:45:17.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.308+0000 7ff9b4bc0000 20 client.4653 trim_cache size 0 max 16384 2024-03-16T20:45:17.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9b4bc0000 1 Processor -- start 2024-03-16T20:45:17.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9b4bc0000 1 -- start start 2024-03-16T20:45:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9b4bc0000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44ccdfdd0 0x55a44cceba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9b4bc0000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44cce06f0 0x55a44cd218f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9b4bc0000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44cce1010 0x55a44cd21e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9b4bc0000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44ccdc180 con 0x55a44ccdfdd0 2024-03-16T20:45:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9b4bc0000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44c7e9940 con 0x55a44cce06f0 2024-03-16T20:45:17.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9b4bc0000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a44caca660 con 0x55a44cce1010 2024-03-16T20:45:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9aaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44ccdfdd0 0x55a44cceba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9aaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44ccdfdd0 0x55a44cceba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41884/0 (socket says 172.21.15.122:41884) 2024-03-16T20:45:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9aaffd640 1 -- 172.21.15.122:0/1304356563 learned_addr learned my addr 172.21.15.122:0/1304356563 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9ab7fe640 1 --2- 172.21.15.122:0/1304356563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44cce1010 0x55a44cd21e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9aa7fc640 1 --2- 172.21.15.122:0/1304356563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44cce06f0 0x55a44cd218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9aaffd640 1 -- 172.21.15.122:0/1304356563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44cce1010 msgr2=0x55a44cd21e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9aaffd640 1 --2- 172.21.15.122:0/1304356563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44cce1010 0x55a44cd21e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9aaffd640 1 -- 172.21.15.122:0/1304356563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44cce06f0 msgr2=0x55a44cd218f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9aaffd640 1 --2- 172.21.15.122:0/1304356563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44cce06f0 0x55a44cd218f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9aaffd640 1 -- 172.21.15.122:0/1304356563 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a44cad7e10 con 0x55a44ccdfdd0 2024-03-16T20:45:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff9aaffd640 1 --2- 172.21.15.122:0/1304356563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44ccdfdd0 0x55a44cceba10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0ef6d0 tx=0x7ff99c11d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cce0ac343128cce server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff990ff9640 1 -- 172.21.15.122:0/1304356563 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c11d4f0 con 0x55a44ccdfdd0 2024-03-16T20:45:17.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff990ff9640 1 -- 172.21.15.122:0/1304356563 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99c0edcd0 con 0x55a44ccdfdd0 2024-03-16T20:45:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.324+0000 7ff990ff9640 1 -- 172.21.15.122:0/1304356563 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0efc30 con 0x55a44ccdfdd0 2024-03-16T20:45:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1304356563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44ccdfdd0 msgr2=0x55a44cceba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/1304356563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44ccdfdd0 0x55a44cceba10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0ef6d0 tx=0x7ff99c11d280 comp rx=0 tx=0).stop 2024-03-16T20:45:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1304356563 shutdown_connections 2024-03-16T20:45:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/1304356563 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44cce1010 0x55a44cd21e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/1304356563 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44cce06f0 0x55a44cd218f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/1304356563 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44ccdfdd0 0x55a44cceba10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1304356563 >> 172.21.15.122:0/1304356563 conn(0x55a44ccdc2c0 msgr2=0x55a44cd2b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1304356563 shutdown_connections 2024-03-16T20:45:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1304356563 wait complete. 2024-03-16T20:45:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 -- start start 2024-03-16T20:45:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 Processor -- start 2024-03-16T20:45:17.337 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44ccdfdd0 0x55a44cd208d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44cce06f0 0x55a44cd20e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44cce1010 0x55a44cd21350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44cb2f540 con 0x55a44ccdfdd0 2024-03-16T20:45:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a44cafcc80 con 0x55a44cce1010 2024-03-16T20:45:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a44cad1f10 con 0x55a44cce06f0 2024-03-16T20:45:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 10 client.0 _ll_register_callbacks cb 0x7ff9a0009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T20:45:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9aaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44ccdfdd0 0x55a44cd208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9aaffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44ccdfdd0 0x55a44cd208d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41896/0 (socket says 172.21.15.122:41896) 2024-03-16T20:45:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9aaffd640 1 -- 172.21.15.122:0/1666458468 learned_addr learned my addr 172.21.15.122:0/1666458468 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9ab7fe640 1 --2- 172.21.15.122:0/1666458468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44cce1010 0x55a44cd21350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9aaffd640 1 -- 172.21.15.122:0/1666458468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44cce06f0 msgr2=0x55a44cd20e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:17.338 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9aaffd640 1 --2- 172.21.15.122:0/1666458468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44cce06f0 0x55a44cd20e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9aaffd640 1 -- 172.21.15.122:0/1666458468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44cce1010 msgr2=0x55a44cd21350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:17.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9aaffd640 1 --2- 172.21.15.122:0/1666458468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44cce1010 0x55a44cd21350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9aaffd640 1 -- 172.21.15.122:0/1666458468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ac143e30 con 0x55a44ccdfdd0 2024-03-16T20:45:17.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9ab7fe640 1 --2- 172.21.15.122:0/1666458468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44cce1010 0x55a44cd21350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:17.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9aaffd640 1 --2- 172.21.15.122:0/1666458468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44ccdfdd0 0x55a44cd208d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff99c097c20 tx=0x7ff99c009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a517a3ead1b917d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:17.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff98a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-16T20:45:17.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff98a7fc640 1 -- 172.21.15.122:0/1666458468 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99c0efe30 con 0x55a44ccdfdd0 2024-03-16T20:45:17.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff98a7fc640 1 -- 172.21.15.122:0/1666458468 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff99c11d4f0 con 0x55a44ccdfdd0 2024-03-16T20:45:17.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1666458468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a44cab7e80 con 0x55a44ccdfdd0 2024-03-16T20:45:17.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff98a7fc640 1 -- 172.21.15.122:0/1666458468 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff99c0f7c50 con 0x55a44ccdfdd0 2024-03-16T20:45:17.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 10 client.4659 fetch_fsmap learned FSMap version 6 2024-03-16T20:45:17.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1666458468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x55a44c828cc0 con 0x55a44ccdfdd0 2024-03-16T20:45:17.339 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff98a7fc640 1 -- 172.21.15.122:0/1666458468 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a44cab7e80 con 0x55a44ccdfdd0 2024-03-16T20:45:17.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff98a7fc640 1 -- 172.21.15.122:0/1666458468 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 6) v1 ==== 2164+0+0 (secure 0 0 0) 0x7ff99c0b6b40 con 0x55a44ccdfdd0 2024-03-16T20:45:17.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 10 client.4659 fetch_fsmap finished waiting for FSMap version 6 2024-03-16T20:45:17.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 10 client.4659 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-16T20:45:17.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 20 client.4659 populate_metadata read hostname 'smithi122' 2024-03-16T20:45:17.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/1666458468 >> [v2:172.21.15.195:6834/1428608180,v1:172.21.15.195:6835/1428608180] conn(0x55a44cd21d10 0x55a44cd81960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 4 client.4659 mds_command: new command op to 4251 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T20:45:17.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1666458468 --> [v2:172.21.15.195:6834/1428608180,v1:172.21.15.195:6835/1428608180] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a44cafafa0 con 0x55a44cd21d10 2024-03-16T20:45:17.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9aa7fc640 1 --2- 172.21.15.122:0/1666458468 >> [v2:172.21.15.195:6834/1428608180,v1:172.21.15.195:6835/1428608180] conn(0x55a44cd21d10 0x55a44cd81960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff98a7fc640 10 client.4659 ms_handle_connect on v2:172.21.15.195:6834/1428608180 2024-03-16T20:45:17.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9aa7fc640 1 --2- 172.21.15.122:0/1666458468 >> [v2:172.21.15.195:6834/1428608180,v1:172.21.15.195:6835/1428608180] conn(0x55a44cd21d10 0x55a44cd81960 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc8f2720e26a94db server_cookie=8789b33c2487d9ea in_seq=0 out_seq=0 2024-03-16T20:45:17.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff98a7fc640 1 -- 172.21.15.122:0/1666458468 <== mds.0 v2:172.21.15.195:6834/1428608180 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55a44cafafa0 con 0x55a44cd21d10 2024-03-16T20:45:17.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff98a7fc640 10 client.4659 handle_command_reply: tid=0 2024-03-16T20:45:17.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.328+0000 7ff9b4bc0000 1 client.4659 shutdown 2024-03-16T20:45:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1666458468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44ccdfdd0 msgr2=0x55a44cd208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/1666458468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44ccdfdd0 0x55a44cd208d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff99c097c20 tx=0x7ff99c009240 comp rx=0 tx=0).stop 2024-03-16T20:45:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1666458468 shutdown_connections 2024-03-16T20:45:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/1666458468 >> [v2:172.21.15.195:6834/1428608180,v1:172.21.15.195:6835/1428608180] conn(0x55a44cd21d10 0x55a44cd81960 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/1666458468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55a44cce1010 0x55a44cd21350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/1666458468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55a44cce06f0 0x55a44cd20e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff98a7fc640 0 client.4659 ms_handle_reset on v2:172.21.15.195:6834/1428608180 2024-03-16T20:45:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b4bc0000 1 --2- 172.21.15.122:0/1666458468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55a44ccdfdd0 0x55a44cd208d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1666458468 >> 172.21.15.122:0/1666458468 conn(0x55a44ccdc2c0 msgr2=0x55a44cd2b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1666458468 shutdown_connections 2024-03-16T20:45:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b4bc0000 1 -- 172.21.15.122:0/1666458468 wait complete. 2024-03-16T20:45:17.341 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b4bc0000 20 client.4659 trim_cache size 0 max 16384 2024-03-16T20:45:17.342 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-16T20:45:17.342 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "ea4c964c-3481-413d-a556-a3fc2950fdd6", 2024-03-16T20:45:17.342 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-03-16T20:45:17.342 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 4251, 2024-03-16T20:45:17.342 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-03-16T20:45:17.342 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-03-16T20:45:17.342 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-03-16T20:45:17.342 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 11.926289085000001, 2024-03-16T20:45:17.342 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 6, 2024-03-16T20:45:17.342 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 21, 2024-03-16T20:45:17.343 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 21, 2024-03-16T20:45:17.343 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 19.022317622999999 2024-03-16T20:45:17.343 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-16T20:45:17.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b1a4e640 1 -- 172.21.15.122:0/1638824327 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff998066e40 msgr2=0x7ff998069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:17.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b1a4e640 1 --2- 172.21.15.122:0/1638824327 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff998066e40 0x7ff998069300 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac141d60 tx=0x7ff9a0008040 comp rx=0 tx=0).stop 2024-03-16T20:45:17.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b1a4e640 1 -- 172.21.15.122:0/1638824327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9ac06b1f0 msgr2=0x7ff9ac1442f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:17.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b1a4e640 1 --2- 172.21.15.122:0/1638824327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9ac06b1f0 0x7ff9ac1442f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff99c0ae040 tx=0x7ff99c01aaa0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b1a4e640 1 -- 172.21.15.122:0/1638824327 shutdown_connections 2024-03-16T20:45:17.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b1a4e640 1 --2- 172.21.15.122:0/1638824327 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff998066e40 0x7ff998069300 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b1a4e640 1 --2- 172.21.15.122:0/1638824327 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9ac1417b0 0x7ff9ac150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b1a4e640 1 --2- 172.21.15.122:0/1638824327 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff9ac06f6f0 0x7ff9ac1e3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b1a4e640 1 --2- 172.21.15.122:0/1638824327 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9ac06b1f0 0x7ff9ac1442f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b1a4e640 1 -- 172.21.15.122:0/1638824327 >> 172.21.15.122:0/1638824327 conn(0x7ff9ac070d30 msgr2=0x7ff9ac13ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:17.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b1a4e640 1 -- 172.21.15.122:0/1638824327 shutdown_connections 2024-03-16T20:45:17.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.332+0000 7ff9b1a4e640 1 -- 172.21.15.122:0/1638824327 wait complete. 2024-03-16T20:45:17.349 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-03-16T20:45:17.373 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-03-16T20:45:17.373 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-03-16T20:45:17.373 INFO:tasks.ceph_fuse:client.0 config is {} 2024-03-16T20:45:17.373 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-16T20:45:17.373 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-16T20:45:17.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:17.374 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-03-16T20:45:17.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:17.379 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link delete ceph-brx 2024-03-16T20:45:17.438 INFO:teuthology.orchestra.run.smithi122.stderr:Cannot find device "ceph-brx" 2024-03-16T20:45:17.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T20:45:17.439 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-03-16T20:45:17.439 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-16T20:45:17.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.600+0000 7fa3bbae8640 1 Processor -- start 2024-03-16T20:45:17.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.600+0000 7fa3bbae8640 1 -- start start 2024-03-16T20:45:17.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.600+0000 7fa3bbae8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b4146a50 0x7fa3b4143460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.600+0000 7fa3bbae8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa3b4071580 0x7fa3b41439a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.600+0000 7fa3bbae8640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3b40b4c50 con 0x7fa3b413d8d0 2024-03-16T20:45:17.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.600+0000 7fa3bbae8640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3b40b2c30 con 0x7fa3b4071580 2024-03-16T20:45:17.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.600+0000 7fa3bbae8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b406bf30 con 0x7fa3b4146a50 2024-03-16T20:45:17.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.600+0000 7fa3ba085640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa3b4071580 0x7fa3b41439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.600+0000 7fa3ba085640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa3b4071580 0x7fa3b41439a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41910/0 (socket says 172.21.15.122:41910) 2024-03-16T20:45:17.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.600+0000 7fa3ba085640 1 -- 172.21.15.122:0/4085036812 learned_addr learned my addr 172.21.15.122:0/4085036812 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:17.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.600+0000 7fa3b9083640 1 --2- 172.21.15.122:0/4085036812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b4146a50 0x7fa3b4143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.600+0000 7fa3ba085640 1 -- 172.21.15.122:0/4085036812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b4146a50 msgr2=0x7fa3b4143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:17.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.600+0000 7fa3ba085640 1 --2- 172.21.15.122:0/4085036812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b4146a50 0x7fa3b4143460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.600+0000 7fa3ba085640 1 -- 172.21.15.122:0/4085036812 >> v1:172.21.15.195:6789/0 conn(0x7fa3b413d8d0 legacy=0x7fa3b406b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:17.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.600+0000 7fa3ba085640 1 -- 172.21.15.122:0/4085036812 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b4146ee0 con 0x7fa3b4071580 2024-03-16T20:45:17.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3ba085640 1 --2- 172.21.15.122:0/4085036812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3b4071580 0x7fa3b41439a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00bb40 tx=0x7fa3ac01ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73be07bdb31c51c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:17.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3b8882640 1 -- 172.21.15.122:0/4085036812 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac0a7b70 con 0x7fa3b4071580 2024-03-16T20:45:17.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3b8882640 1 -- 172.21.15.122:0/4085036812 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ac0a7cd0 con 0x7fa3b4071580 2024-03-16T20:45:17.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3b8882640 1 -- 172.21.15.122:0/4085036812 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac0af660 con 0x7fa3b4071580 2024-03-16T20:45:17.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 -- 172.21.15.122:0/4085036812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3b4071580 msgr2=0x7fa3b41439a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:17.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 --2- 172.21.15.122:0/4085036812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3b4071580 0x7fa3b41439a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac00bb40 tx=0x7fa3ac01ec10 comp rx=0 tx=0).stop 2024-03-16T20:45:17.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 -- 172.21.15.122:0/4085036812 shutdown_connections 2024-03-16T20:45:17.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 --2- 172.21.15.122:0/4085036812 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3b4071580 0x7fa3b41439a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 --2- 172.21.15.122:0/4085036812 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b4146a50 0x7fa3b4143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 -- 172.21.15.122:0/4085036812 >> 172.21.15.122:0/4085036812 conn(0x7fa3b40b8860 msgr2=0x7fa3b40b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:17.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 -- 172.21.15.122:0/4085036812 shutdown_connections 2024-03-16T20:45:17.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 -- 172.21.15.122:0/4085036812 wait complete. 2024-03-16T20:45:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 Processor -- start 2024-03-16T20:45:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 -- start start 2024-03-16T20:45:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b4071580 0x7fa3b4142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3b413d8d0 0x7fa3b41de930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3b41dee70 0x7fa3b4137830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b413dea0 con 0x7fa3b413d8d0 2024-03-16T20:45:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b4146860 con 0x7fa3b41dee70 2024-03-16T20:45:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b406b820 con 0x7fa3b4071580 2024-03-16T20:45:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3b9083640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3b413d8d0 0x7fa3b41de930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3b9083640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3b413d8d0 0x7fa3b41de930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:41918/0 (socket says 172.21.15.122:41918) 2024-03-16T20:45:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3b9083640 1 -- 172.21.15.122:0/425572926 learned_addr learned my addr 172.21.15.122:0/425572926 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3b9884640 1 --2- 172.21.15.122:0/425572926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b4071580 0x7fa3b4142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3ba085640 1 --2- 172.21.15.122:0/425572926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3b41dee70 0x7fa3b4137830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3b9083640 1 -- 172.21.15.122:0/425572926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b4071580 msgr2=0x7fa3b4142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3b9083640 1 --2- 172.21.15.122:0/425572926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b4071580 0x7fa3b4142680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3b9083640 1 -- 172.21.15.122:0/425572926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3b41dee70 msgr2=0x7fa3b4137830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:17.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3b9083640 1 --2- 172.21.15.122:0/425572926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3b41dee70 0x7fa3b4137830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3b9083640 1 -- 172.21.15.122:0/425572926 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ac00b770 con 0x7fa3b413d8d0 2024-03-16T20:45:17.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3b9083640 1 --2- 172.21.15.122:0/425572926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3b413d8d0 0x7fa3b41de930 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8007ef0 tx=0x7fa3a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:17.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3b27fc640 1 -- 172.21.15.122:0/425572926 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8014620 con 0x7fa3b413d8d0 2024-03-16T20:45:17.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3b27fc640 1 -- 172.21.15.122:0/425572926 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a8014780 con 0x7fa3b413d8d0 2024-03-16T20:45:17.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 -- 172.21.15.122:0/425572926 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b4146580 con 0x7fa3b413d8d0 2024-03-16T20:45:17.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3b27fc640 1 -- 172.21.15.122:0/425572926 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8014b30 con 0x7fa3b413d8d0 2024-03-16T20:45:17.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.604+0000 7fa3bbae8640 1 -- 172.21.15.122:0/425572926 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b4145b00 con 0x7fa3b413d8d0 2024-03-16T20:45:17.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.608+0000 7fa3bbae8640 1 -- 172.21.15.122:0/425572926 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3b4071d10 con 0x7fa3b413d8d0 2024-03-16T20:45:17.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.608+0000 7fa3b27fc640 1 -- 172.21.15.122:0/425572926 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fa3a8014c90 con 0x7fa3b413d8d0 2024-03-16T20:45:17.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.608+0000 7fa3b27fc640 1 --2- 172.21.15.122:0/425572926 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa3a0066e40 0x7fa3a0069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:17.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.608+0000 7fa3b27fc640 1 -- 172.21.15.122:0/425572926 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3a8085430 con 0x7fa3b413d8d0 2024-03-16T20:45:17.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.608+0000 7fa3b9884640 1 --2- 172.21.15.122:0/425572926 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa3a0066e40 0x7fa3a0069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:17.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.608+0000 7fa3b9884640 1 --2- 172.21.15.122:0/425572926 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa3a0066e40 0x7fa3a0069300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4007fd0 tx=0x7fa3a4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:17.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.612+0000 7fa3b27fc640 1 -- 172.21.15.122:0/425572926 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa3a8052220 con 0x7fa3b413d8d0 2024-03-16T20:45:17.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.792+0000 7fa3bbae8640 1 -- 172.21.15.122:0/425572926 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa3b413dfe0 con 0x7fa3b413d8d0 2024-03-16T20:45:17.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.792+0000 7fa3b27fc640 1 -- 172.21.15.122:0/425572926 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa3a80558e0 con 0x7fa3b413d8d0 2024-03-16T20:45:17.795 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-16T20:45:17.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.792+0000 7fa3bbae8640 1 -- 172.21.15.122:0/425572926 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa3a0066e40 msgr2=0x7fa3a0069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:17.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.792+0000 7fa3bbae8640 1 --2- 172.21.15.122:0/425572926 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa3a0066e40 0x7fa3a0069300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4007fd0 tx=0x7fa3a4009040 comp rx=0 tx=0).stop 2024-03-16T20:45:17.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.792+0000 7fa3bbae8640 1 -- 172.21.15.122:0/425572926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3b413d8d0 msgr2=0x7fa3b41de930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:17.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.792+0000 7fa3bbae8640 1 --2- 172.21.15.122:0/425572926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3b413d8d0 0x7fa3b41de930 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa3a8007ef0 tx=0x7fa3a800e4b0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.792+0000 7fa3bbae8640 1 -- 172.21.15.122:0/425572926 shutdown_connections 2024-03-16T20:45:17.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.792+0000 7fa3bbae8640 1 --2- 172.21.15.122:0/425572926 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa3a0066e40 0x7fa3a0069300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.792+0000 7fa3bbae8640 1 --2- 172.21.15.122:0/425572926 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa3b41dee70 0x7fa3b4137830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.792+0000 7fa3bbae8640 1 --2- 172.21.15.122:0/425572926 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa3b413d8d0 0x7fa3b41de930 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.792+0000 7fa3bbae8640 1 --2- 172.21.15.122:0/425572926 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa3b4071580 0x7fa3b4142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:17.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.792+0000 7fa3bbae8640 1 -- 172.21.15.122:0/425572926 >> 172.21.15.122:0/425572926 conn(0x7fa3b40b8860 msgr2=0x7fa3b40b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:17.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.792+0000 7fa3bbae8640 1 -- 172.21.15.122:0/425572926 shutdown_connections 2024-03-16T20:45:17.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:17.792+0000 7fa3bbae8640 1 -- 172.21.15.122:0/425572926 wait complete. 2024-03-16T20:45:17.812 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-16T20:45:17.812 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-16T20:45:17.812 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-03-16T20:45:17.812 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-16T20:45:17.812 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-16T20:45:17.812 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-16T20:45:17.812 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-16T20:45:17.813 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-16T20:45:17.813 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-16T20:45:17.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:17.813 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-03-16T20:45:17.819 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T20:45:17.819 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T20:45:17.819 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T20:45:17.820 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-03-16T20:45:17.820 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-03-16T20:45:17.820 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-03-16T20:45:17.820 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-16T20:45:17.820 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-03-16T20:45:17.820 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-03-16T20:45:17.820 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-16T20:45:17.820 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-03-16T20:45:17.820 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-03-16T20:45:17.820 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-16T20:45:17.820 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-03-16T20:45:17.820 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-16T20:45:17.820 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-03-16T20:45:17.820 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-16T20:45:17.821 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 42526sec preferred_lft 42526sec 2024-03-16T20:45:17.821 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-03-16T20:45:17.821 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-03-16T20:45:17.821 INFO:teuthology.orchestra.run.smithi122.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-16T20:45:17.821 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 02:42:0d:60:22:73 brd ff:ff:ff:ff:ff:ff 2024-03-16T20:45:17.821 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-16T20:45:17.821 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-03-16T20:45:17.821 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-16T20:45:17.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:17.821 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec bash -c ' 2024-03-16T20:45:17.822 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-03-16T20:45:17.822 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add name ceph-brx type bridge 2024-03-16T20:45:17.822 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip addr flush dev ceph-brx 2024-03-16T20:45:17.822 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set ceph-brx up 2024-03-16T20:45:17.822 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-16T20:45:17.822 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-03-16T20:45:17.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:17.892 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-16T20:45:17.941 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-03-16T20:45:17.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:17.942 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-03-16T20:45:18.004 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-03-16T20:45:18.004 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-16T20:45:18.004 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-16T20:45:18.004 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-16T20:45:18.004 INFO:teuthology.orchestra.run.smithi122.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-16T20:45:18.004 INFO:teuthology.orchestra.run.smithi122.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-16T20:45:18.005 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-16T20:45:18.005 INFO:teuthology.orchestra.run.smithi122.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-16T20:45:18.005 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-16T20:45:18.005 INFO:teuthology.orchestra.run.smithi122.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-16T20:45:18.005 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-16T20:45:18.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:18.005 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec bash -c ' 2024-03-16T20:45:18.005 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-03-16T20:45:18.005 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-16T20:45:18.005 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-16T20:45:18.005 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-16T20:45:18.006 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-03-16T20:45:18.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:18.074 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-03-16T20:45:18.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:18.123 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list-id 2024-03-16T20:45:18.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:18.170 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec bash -c ' 2024-03-16T20:45:18.170 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-03-16T20:45:18.170 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T20:45:18.170 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-16T20:45:18.171 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-03-16T20:45:18.238 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-16T20:45:18.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:18.239 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec bash -c ' 2024-03-16T20:45:18.239 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-03-16T20:45:18.239 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-16T20:45:18.239 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-16T20:45:18.239 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-16T20:45:18.239 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-16T20:45:18.239 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-16T20:45:18.239 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-03-16T20:45:18.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:18.345 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec bash -c ' 2024-03-16T20:45:18.345 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-03-16T20:45:18.345 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-03-16T20:45:18.345 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set dev brx.0 master ceph-brx 2024-03-16T20:45:18.345 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-03-16T20:45:18.363 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-16T20:45:18.363 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T20:45:18.363 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-16T20:45:18.410 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-16T20:45:18.410 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T20:45:18.410 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-16T20:45:18.457 DEBUG:teuthology.orchestra.run.smithi122:> sudo modprobe fuse 2024-03-16T20:45:18.508 DEBUG:teuthology.orchestra.run.smithi122:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T20:45:18.555 INFO:teuthology.orchestra.run.smithi122.stdout:/sys 2024-03-16T20:45:18.555 INFO:teuthology.orchestra.run.smithi122.stdout:/proc 2024-03-16T20:45:18.555 INFO:teuthology.orchestra.run.smithi122.stdout:/dev 2024-03-16T20:45:18.555 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/pts 2024-03-16T20:45:18.555 INFO:teuthology.orchestra.run.smithi122.stdout:/run 2024-03-16T20:45:18.555 INFO:teuthology.orchestra.run.smithi122.stdout:/ 2024-03-16T20:45:18.555 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/security 2024-03-16T20:45:18.555 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/shm 2024-03-16T20:45:18.555 INFO:teuthology.orchestra.run.smithi122.stdout:/run/lock 2024-03-16T20:45:18.556 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/cgroup 2024-03-16T20:45:18.556 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/pstore 2024-03-16T20:45:18.556 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/bpf 2024-03-16T20:45:18.556 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T20:45:18.556 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/mqueue 2024-03-16T20:45:18.556 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/hugepages 2024-03-16T20:45:18.556 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/debug 2024-03-16T20:45:18.556 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/tracing 2024-03-16T20:45:18.556 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/fs/nfsd 2024-03-16T20:45:18.556 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/fuse/connections 2024-03-16T20:45:18.556 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/config 2024-03-16T20:45:18.556 INFO:teuthology.orchestra.run.smithi122.stdout:/run/credentials/systemd-sysusers.service 2024-03-16T20:45:18.556 INFO:teuthology.orchestra.run.smithi122.stdout:/run/rpc_pipefs 2024-03-16T20:45:18.556 INFO:teuthology.orchestra.run.smithi122.stdout:/run/user/1000 2024-03-16T20:45:18.556 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph 2024-03-16T20:45:18.556 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-0 2024-03-16T20:45:18.556 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-1 2024-03-16T20:45:18.557 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-2 2024-03-16T20:45:18.557 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-3 2024-03-16T20:45:18.557 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns 2024-03-16T20:45:18.557 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T20:45:18.557 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T20:45:18.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:18.558 DEBUG:teuthology.orchestra.run.smithi122:> ls /sys/fs/fuse/connections 2024-03-16T20:45:18.605 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-16T20:45:18.605 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-16T20:45:18.651 DEBUG:teuthology.orchestra.run.smithi122:> sudo modprobe fuse 2024-03-16T20:45:18.665 DEBUG:teuthology.orchestra.run.smithi122:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T20:45:18.710 INFO:teuthology.orchestra.run.smithi122.stdout:/sys 2024-03-16T20:45:18.711 INFO:teuthology.orchestra.run.smithi122.stdout:/proc 2024-03-16T20:45:18.711 INFO:teuthology.orchestra.run.smithi122.stdout:/dev 2024-03-16T20:45:18.711 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/pts 2024-03-16T20:45:18.711 INFO:teuthology.orchestra.run.smithi122.stdout:/run 2024-03-16T20:45:18.711 INFO:teuthology.orchestra.run.smithi122.stdout:/ 2024-03-16T20:45:18.711 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/security 2024-03-16T20:45:18.711 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/shm 2024-03-16T20:45:18.711 INFO:teuthology.orchestra.run.smithi122.stdout:/run/lock 2024-03-16T20:45:18.711 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/cgroup 2024-03-16T20:45:18.711 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/pstore 2024-03-16T20:45:18.711 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/bpf 2024-03-16T20:45:18.711 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T20:45:18.711 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/mqueue 2024-03-16T20:45:18.711 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/hugepages 2024-03-16T20:45:18.711 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/debug 2024-03-16T20:45:18.711 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/tracing 2024-03-16T20:45:18.712 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/fs/nfsd 2024-03-16T20:45:18.712 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/fuse/connections 2024-03-16T20:45:18.712 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/config 2024-03-16T20:45:18.712 INFO:teuthology.orchestra.run.smithi122.stdout:/run/credentials/systemd-sysusers.service 2024-03-16T20:45:18.712 INFO:teuthology.orchestra.run.smithi122.stdout:/run/rpc_pipefs 2024-03-16T20:45:18.712 INFO:teuthology.orchestra.run.smithi122.stdout:/run/user/1000 2024-03-16T20:45:18.712 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph 2024-03-16T20:45:18.712 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-0 2024-03-16T20:45:18.712 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-1 2024-03-16T20:45:18.712 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-2 2024-03-16T20:45:18.712 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-3 2024-03-16T20:45:18.712 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns 2024-03-16T20:45:18.712 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T20:45:18.712 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T20:45:18.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:18.713 DEBUG:teuthology.orchestra.run.smithi122:> ls /sys/fs/fuse/connections 2024-03-16T20:45:18.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi122.stderr:2024-03-16T20:45:18.732+0000 7f0b81862580 -1 init, newargv = 0x557bf367e110 newargc=15 2024-03-16T20:45:18.737 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi122.stderr:ceph-fuse[34631]: starting ceph client 2024-03-16T20:45:18.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi122.stderr:ceph-fuse[34631]: starting fuse 2024-03-16T20:45:19.842 DEBUG:teuthology.orchestra.run.smithi122:> sudo modprobe fuse 2024-03-16T20:45:19.859 DEBUG:teuthology.orchestra.run.smithi122:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T20:45:19.907 INFO:teuthology.orchestra.run.smithi122.stdout:/sys 2024-03-16T20:45:19.907 INFO:teuthology.orchestra.run.smithi122.stdout:/proc 2024-03-16T20:45:19.907 INFO:teuthology.orchestra.run.smithi122.stdout:/dev 2024-03-16T20:45:19.907 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/pts 2024-03-16T20:45:19.907 INFO:teuthology.orchestra.run.smithi122.stdout:/run 2024-03-16T20:45:19.907 INFO:teuthology.orchestra.run.smithi122.stdout:/ 2024-03-16T20:45:19.907 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/security 2024-03-16T20:45:19.908 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/shm 2024-03-16T20:45:19.908 INFO:teuthology.orchestra.run.smithi122.stdout:/run/lock 2024-03-16T20:45:19.908 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/cgroup 2024-03-16T20:45:19.908 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/pstore 2024-03-16T20:45:19.908 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/bpf 2024-03-16T20:45:19.908 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T20:45:19.908 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/mqueue 2024-03-16T20:45:19.908 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/hugepages 2024-03-16T20:45:19.908 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/debug 2024-03-16T20:45:19.908 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/tracing 2024-03-16T20:45:19.908 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/fs/nfsd 2024-03-16T20:45:19.908 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/fuse/connections 2024-03-16T20:45:19.908 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/config 2024-03-16T20:45:19.908 INFO:teuthology.orchestra.run.smithi122.stdout:/run/credentials/systemd-sysusers.service 2024-03-16T20:45:19.908 INFO:teuthology.orchestra.run.smithi122.stdout:/run/rpc_pipefs 2024-03-16T20:45:19.909 INFO:teuthology.orchestra.run.smithi122.stdout:/run/user/1000 2024-03-16T20:45:19.909 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph 2024-03-16T20:45:19.909 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-0 2024-03-16T20:45:19.909 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-1 2024-03-16T20:45:19.909 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-2 2024-03-16T20:45:19.909 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-3 2024-03-16T20:45:19.909 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns 2024-03-16T20:45:19.909 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T20:45:19.909 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T20:45:19.909 INFO:teuthology.orchestra.run.smithi122.stdout:/home/ubuntu/cephtest/mnt.0 2024-03-16T20:45:19.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:19.910 DEBUG:teuthology.orchestra.run.smithi122:> ls /sys/fs/fuse/connections 2024-03-16T20:45:19.958 INFO:teuthology.orchestra.run.smithi122.stdout:45 2024-03-16T20:45:19.959 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-16T20:45:19.959 DEBUG:teuthology.orchestra.run.smithi122:> sudo stdin-killer -- python3 -c ' 2024-03-16T20:45:19.959 DEBUG:teuthology.orchestra.run.smithi122:> import glob 2024-03-16T20:45:19.959 DEBUG:teuthology.orchestra.run.smithi122:> import re 2024-03-16T20:45:19.959 DEBUG:teuthology.orchestra.run.smithi122:> import os 2024-03-16T20:45:19.959 DEBUG:teuthology.orchestra.run.smithi122:> import subprocess 2024-03-16T20:45:19.959 DEBUG:teuthology.orchestra.run.smithi122:> 2024-03-16T20:45:19.959 DEBUG:teuthology.orchestra.run.smithi122:> def _find_admin_socket(client_name): 2024-03-16T20:45:19.959 DEBUG:teuthology.orchestra.run.smithi122:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-16T20:45:19.959 DEBUG:teuthology.orchestra.run.smithi122:> files = glob.glob(asok_path) 2024-03-16T20:45:19.959 DEBUG:teuthology.orchestra.run.smithi122:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-16T20:45:19.959 DEBUG:teuthology.orchestra.run.smithi122:> 2024-03-16T20:45:19.960 DEBUG:teuthology.orchestra.run.smithi122:> # Given a non-glob path, it better be there 2024-03-16T20:45:19.960 DEBUG:teuthology.orchestra.run.smithi122:> if "*" not in asok_path: 2024-03-16T20:45:19.960 DEBUG:teuthology.orchestra.run.smithi122:> assert(len(files) == 1) 2024-03-16T20:45:19.960 DEBUG:teuthology.orchestra.run.smithi122:> return files[0] 2024-03-16T20:45:19.960 DEBUG:teuthology.orchestra.run.smithi122:> 2024-03-16T20:45:19.960 DEBUG:teuthology.orchestra.run.smithi122:> for f in files: 2024-03-16T20:45:19.960 DEBUG:teuthology.orchestra.run.smithi122:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-16T20:45:19.960 DEBUG:teuthology.orchestra.run.smithi122:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-16T20:45:19.960 DEBUG:teuthology.orchestra.run.smithi122:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-16T20:45:19.960 DEBUG:teuthology.orchestra.run.smithi122:> contents = proc_f.read() 2024-03-16T20:45:19.960 DEBUG:teuthology.orchestra.run.smithi122:> if mountpoint in contents: 2024-03-16T20:45:19.960 DEBUG:teuthology.orchestra.run.smithi122:> return f 2024-03-16T20:45:19.960 DEBUG:teuthology.orchestra.run.smithi122:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-16T20:45:19.960 DEBUG:teuthology.orchestra.run.smithi122:> 2024-03-16T20:45:19.960 DEBUG:teuthology.orchestra.run.smithi122:> print(_find_admin_socket("client.0")) 2024-03-16T20:45:19.960 DEBUG:teuthology.orchestra.run.smithi122:> ' 2024-03-16T20:45:20.081 INFO:teuthology.orchestra.run.smithi122.stdout:/var/run/ceph/ceph-client.0.34631.asok 2024-03-16T20:45:20.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T20:45:20.093 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.34631.asok 2024-03-16T20:45:20.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:20.093 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.34631.asok status 2024-03-16T20:45:20.241 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-16T20:45:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "metadata": { 2024-03-16T20:45:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_sha1": "aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b", 2024-03-16T20:45:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_version": "ceph version 19.0.0-1681-gaba3c1d2 (aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b) squid (dev)", 2024-03-16T20:45:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "entity_id": "0", 2024-03-16T20:45:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "hostname": "smithi122", 2024-03-16T20:45:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-16T20:45:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "pid": "34631", 2024-03-16T20:45:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "root": "/" 2024-03-16T20:45:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "dentry_count": 0, 2024-03-16T20:45:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "dentry_pinned_count": 0, 2024-03-16T20:45:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 4671, 2024-03-16T20:45:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "inst": { 2024-03-16T20:45:20.241 INFO:teuthology.orchestra.run.smithi122.stdout: "name": { 2024-03-16T20:45:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "client", 2024-03-16T20:45:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "num": 4671 2024-03-16T20:45:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": { 2024-03-16T20:45:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v1", 2024-03-16T20:45:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "192.168.0.1:0", 2024-03-16T20:45:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 429238066 2024-03-16T20:45:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": { 2024-03-16T20:45:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v1", 2024-03-16T20:45:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "192.168.0.1:0", 2024-03-16T20:45:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 429238066 2024-03-16T20:45:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "inst_str": "client.4671 192.168.0.1:0/429238066", 2024-03-16T20:45:20.242 INFO:teuthology.orchestra.run.smithi122.stdout: "addr_str": "192.168.0.1:0/429238066", 2024-03-16T20:45:20.243 INFO:teuthology.orchestra.run.smithi122.stdout: "inode_count": 1, 2024-03-16T20:45:20.243 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_epoch": 6, 2024-03-16T20:45:20.243 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_epoch": 21, 2024-03-16T20:45:20.243 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_epoch_barrier": 0, 2024-03-16T20:45:20.243 INFO:teuthology.orchestra.run.smithi122.stdout: "blocklisted": false, 2024-03-16T20:45:20.243 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs" 2024-03-16T20:45:20.243 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-16T20:45:20.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:20.255 DEBUG:teuthology.orchestra.run.smithi122:> stat --file-system '--printf=%T 2024-03-16T20:45:20.255 DEBUG:teuthology.orchestra.run.smithi122:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-16T20:45:20.260 INFO:teuthology.orchestra.run.smithi122.stdout:fuseblk 2024-03-16T20:45:20.261 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-16T20:45:20.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:20.261 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-16T20:45:20.312 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-03-16T20:45:20.327 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:20.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f141dd3c640 1 Processor -- start 2024-03-16T20:45:20.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f141dd3c640 1 -- start start 2024-03-16T20:45:20.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f141dd3c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f14180b77b0 0x7f141813f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:20.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f141dd3c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14180b3830 0x7f14180b7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:20.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f141dd3c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14180b7b90 con 0x7f14180b8500 2024-03-16T20:45:20.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f141dd3c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14180b3500 con 0x7f14180b77b0 2024-03-16T20:45:20.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f141dd3c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14180b70c0 con 0x7f14180b3830 2024-03-16T20:45:20.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f14177fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f14180b77b0 0x7f141813f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:20.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f1417fff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f14180b8500 0x7f141806af90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:51698/0 (socket says 172.21.15.122:51698) 2024-03-16T20:45:20.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f1417fff640 1 -- 172.21.15.122:0/3274046119 learned_addr learned my addr 172.21.15.122:0/3274046119 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:20.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f140ffff640 1 --2- 172.21.15.122:0/3274046119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14180b3830 0x7f14180b7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:20.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f14177fe640 1 -- 172.21.15.122:0/3274046119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14180b3830 msgr2=0x7f14180b7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:20.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f14177fe640 1 --2- 172.21.15.122:0/3274046119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14180b3830 0x7f14180b7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f14177fe640 1 -- 172.21.15.122:0/3274046119 >> v1:172.21.15.122:6789/0 conn(0x7f14180b8500 legacy=0x7f141806af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:20.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f14177fe640 1 -- 172.21.15.122:0/3274046119 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14180b2e40 con 0x7f14180b77b0 2024-03-16T20:45:20.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f14177fe640 1 --2- 172.21.15.122:0/3274046119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14180b77b0 0x7f141813f1c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f14080097c0 tx=0x7f140801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8086a3118c5f66f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:20.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f1416ffd640 1 -- 172.21.15.122:0/3274046119 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14080ab600 con 0x7f14180b77b0 2024-03-16T20:45:20.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f1416ffd640 1 -- 172.21.15.122:0/3274046119 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14080a7050 con 0x7f14180b77b0 2024-03-16T20:45:20.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.476+0000 7f1416ffd640 1 -- 172.21.15.122:0/3274046119 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14080ab980 con 0x7f14180b77b0 2024-03-16T20:45:20.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f141dd3c640 1 -- 172.21.15.122:0/3274046119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14180b77b0 msgr2=0x7f141813f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:20.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f141dd3c640 1 --2- 172.21.15.122:0/3274046119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14180b77b0 0x7f141813f1c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f14080097c0 tx=0x7f140801c3e0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f141dd3c640 1 -- 172.21.15.122:0/3274046119 shutdown_connections 2024-03-16T20:45:20.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f141dd3c640 1 --2- 172.21.15.122:0/3274046119 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14180b3830 0x7f14180b7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f141dd3c640 1 --2- 172.21.15.122:0/3274046119 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14180b77b0 0x7f141813f1c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f141dd3c640 1 -- 172.21.15.122:0/3274046119 >> 172.21.15.122:0/3274046119 conn(0x7f1418070d30 msgr2=0x7f1418071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:20.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f141dd3c640 1 -- 172.21.15.122:0/3274046119 shutdown_connections 2024-03-16T20:45:20.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f141dd3c640 1 -- 172.21.15.122:0/3274046119 wait complete. 2024-03-16T20:45:20.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f141dd3c640 1 Processor -- start 2024-03-16T20:45:20.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f141dd3c640 1 -- start start 2024-03-16T20:45:20.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f141dd3c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14180b3830 0x7f1418133d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:20.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f141dd3c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14180b77b0 0x7f141813c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:20.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f141dd3c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14180b8500 0x7f141812e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:20.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f141dd3c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141813d940 con 0x7f14180b8500 2024-03-16T20:45:20.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f141dd3c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1418144e20 con 0x7f14180b3830 2024-03-16T20:45:20.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f141dd3c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141813e4f0 con 0x7f14180b77b0 2024-03-16T20:45:20.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f1417fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14180b8500 0x7f141812e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:20.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f14177fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14180b3830 0x7f1418133d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:20.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f140ffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14180b77b0 0x7f141813c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:20.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f1417fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14180b8500 0x7f141812e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50856/0 (socket says 172.21.15.122:50856) 2024-03-16T20:45:20.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f1417fff640 1 -- 172.21.15.122:0/776480980 learned_addr learned my addr 172.21.15.122:0/776480980 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:20.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f14177fe640 1 --2- 172.21.15.122:0/776480980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14180b3830 0x7f1418133d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.122:34348/0 (socket says 172.21.15.122:34348) 2024-03-16T20:45:20.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f140ffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14180b77b0 0x7f141813c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:46298/0 (socket says 172.21.15.122:46298) 2024-03-16T20:45:20.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f1417fff640 1 -- 172.21.15.122:0/776480980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14180b77b0 msgr2=0x7f141813c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:20.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f1417fff640 1 --2- 172.21.15.122:0/776480980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14180b77b0 0x7f141813c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f1417fff640 1 -- 172.21.15.122:0/776480980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14180b3830 msgr2=0x7f1418133d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:20.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f1417fff640 1 --2- 172.21.15.122:0/776480980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14180b3830 0x7f1418133d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f1417fff640 1 -- 172.21.15.122:0/776480980 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1408009420 con 0x7f14180b8500 2024-03-16T20:45:20.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f1417fff640 1 --2- 172.21.15.122:0/776480980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14180b8500 0x7f141812e7b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f141000cbc0 tx=0x7f141000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:20.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.480+0000 7f1414ff9640 1 -- 172.21.15.122:0/776480980 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141000ad40 con 0x7f14180b8500 2024-03-16T20:45:20.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.484+0000 7f1414ff9640 1 -- 172.21.15.122:0/776480980 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1410012400 con 0x7f14180b8500 2024-03-16T20:45:20.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.484+0000 7f1414ff9640 1 -- 172.21.15.122:0/776480980 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141000aea0 con 0x7f14180b8500 2024-03-16T20:45:20.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.484+0000 7f141dd3c640 1 -- 172.21.15.122:0/776480980 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141813e7d0 con 0x7f14180b8500 2024-03-16T20:45:20.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.484+0000 7f141dd3c640 1 -- 172.21.15.122:0/776480980 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1418138d20 con 0x7f14180b8500 2024-03-16T20:45:20.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.484+0000 7f141dd3c640 1 -- 172.21.15.122:0/776480980 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141813ea10 con 0x7f14180b8500 2024-03-16T20:45:20.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.484+0000 7f1414ff9640 1 -- 172.21.15.122:0/776480980 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f1410007070 con 0x7f14180b8500 2024-03-16T20:45:20.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.484+0000 7f1414ff9640 1 --2- 172.21.15.122:0/776480980 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f13f8066e40 0x7f13f8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:20.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.484+0000 7f1414ff9640 1 -- 172.21.15.122:0/776480980 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1410084980 con 0x7f14180b8500 2024-03-16T20:45:20.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.484+0000 7f14177fe640 1 --2- 172.21.15.122:0/776480980 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f13f8066e40 0x7f13f8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:20.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.484+0000 7f14177fe640 1 --2- 172.21.15.122:0/776480980 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f13f8066e40 0x7f13f8069300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f14080ad000 tx=0x7f1408008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:20.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.488+0000 7f1414ff9640 1 -- 172.21.15.122:0/776480980 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f141813ea10 con 0x7f14180b8500 2024-03-16T20:45:20.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.648+0000 7f141dd3c640 1 -- 172.21.15.122:0/776480980 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14180ba6b0 con 0x7f14180b8500 2024-03-16T20:45:20.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.652+0000 7f1414ff9640 1 -- 172.21.15.122:0/776480980 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f1410017090 con 0x7f14180b8500 2024-03-16T20:45:20.655 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:20.655 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:06.404479+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.652+0000 7f141dd3c640 1 -- 172.21.15.122:0/776480980 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f13f8066e40 msgr2=0x7f13f8069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.652+0000 7f141dd3c640 1 --2- 172.21.15.122:0/776480980 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f13f8066e40 0x7f13f8069300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f14080ad000 tx=0x7f1408008040 comp rx=0 tx=0).stop 2024-03-16T20:45:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.652+0000 7f141dd3c640 1 -- 172.21.15.122:0/776480980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14180b8500 msgr2=0x7f141812e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.652+0000 7f141dd3c640 1 --2- 172.21.15.122:0/776480980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14180b8500 0x7f141812e7b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f141000cbc0 tx=0x7f141000a430 comp rx=0 tx=0).stop 2024-03-16T20:45:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.652+0000 7f141dd3c640 1 -- 172.21.15.122:0/776480980 shutdown_connections 2024-03-16T20:45:20.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.652+0000 7f141dd3c640 1 --2- 172.21.15.122:0/776480980 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f13f8066e40 0x7f13f8069300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.652+0000 7f141dd3c640 1 --2- 172.21.15.122:0/776480980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14180b8500 0x7f141812e7b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.652+0000 7f141dd3c640 1 --2- 172.21.15.122:0/776480980 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f14180b77b0 0x7f141813c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.652+0000 7f141dd3c640 1 --2- 172.21.15.122:0/776480980 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14180b3830 0x7f1418133d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.652+0000 7f141dd3c640 1 -- 172.21.15.122:0/776480980 >> 172.21.15.122:0/776480980 conn(0x7f1418070d30 msgr2=0x7f1418135540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.652+0000 7f141dd3c640 1 -- 172.21.15.122:0/776480980 shutdown_connections 2024-03-16T20:45:20.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.652+0000 7f141dd3c640 1 -- 172.21.15.122:0/776480980 wait complete. 2024-03-16T20:45:20.675 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T20:44:52.653523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T20:45:02.758515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T20:45:03.710783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T20:45:20.675 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T20:45:20.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.796+0000 7fe56a134640 1 Processor -- start 2024-03-16T20:45:20.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.796+0000 7fe56a134640 1 -- start start 2024-03-16T20:45:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.800+0000 7fe56a134640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe56406ddb0 0x7fe56409b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.800+0000 7fe56a134640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe5641475f0 0x7fe56414e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.800+0000 7fe56a134640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe56409df60 con 0x7fe5640b3370 2024-03-16T20:45:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.800+0000 7fe56a134640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5640b4a50 con 0x7fe56406ddb0 2024-03-16T20:45:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.800+0000 7fe56a134640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5640b3750 con 0x7fe5641475f0 2024-03-16T20:45:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.800+0000 7fe562ffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe56406ddb0 0x7fe56409b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:20.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.800+0000 7fe5637fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fe5640b3370 0x7fe564146ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:51712/0 (socket says 172.21.15.122:51712) 2024-03-16T20:45:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.800+0000 7fe5637fe640 1 -- 172.21.15.122:0/3666884684 learned_addr learned my addr 172.21.15.122:0/3666884684 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.800+0000 7fe562ffd640 1 -- 172.21.15.122:0/3666884684 >> v2:172.21.15.195:3300/0 conn(0x7fe5641475f0 msgr2=0x7fe56414e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.800+0000 7fe562ffd640 1 --2- 172.21.15.122:0/3666884684 >> v2:172.21.15.195:3300/0 conn(0x7fe5641475f0 0x7fe56414e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.800+0000 7fe562ffd640 1 -- 172.21.15.122:0/3666884684 >> v1:172.21.15.122:6789/0 conn(0x7fe5640b3370 legacy=0x7fe564146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.800+0000 7fe5627fc640 1 -- 172.21.15.122:0/3666884684 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767594169 0 0) 0x7fe56409df60 con 0x7fe5640b3370 2024-03-16T20:45:20.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.800+0000 7fe562ffd640 1 -- 172.21.15.122:0/3666884684 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5640b7fe0 con 0x7fe56406ddb0 2024-03-16T20:45:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.800+0000 7fe562ffd640 1 --2- 172.21.15.122:0/3666884684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe56406ddb0 0x7fe56409b7c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe5640b6cf0 tx=0x7fe55801e7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463a4ec9232cefbc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.800+0000 7fe5627fc640 1 -- 172.21.15.122:0/3666884684 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5580a7860 con 0x7fe56406ddb0 2024-03-16T20:45:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.800+0000 7fe5627fc640 1 -- 172.21.15.122:0/3666884684 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5580ab070 con 0x7fe56406ddb0 2024-03-16T20:45:20.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.800+0000 7fe5627fc640 1 -- 172.21.15.122:0/3666884684 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5580a7be0 con 0x7fe56406ddb0 2024-03-16T20:45:20.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 -- 172.21.15.122:0/3666884684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe56406ddb0 msgr2=0x7fe56409b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:20.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 --2- 172.21.15.122:0/3666884684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe56406ddb0 0x7fe56409b7c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe5640b6cf0 tx=0x7fe55801e7f0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 -- 172.21.15.122:0/3666884684 shutdown_connections 2024-03-16T20:45:20.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 --2- 172.21.15.122:0/3666884684 >> v2:172.21.15.195:3300/0 conn(0x7fe5641475f0 0x7fe56414e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 --2- 172.21.15.122:0/3666884684 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe56406ddb0 0x7fe56409b7c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 -- 172.21.15.122:0/3666884684 >> 172.21.15.122:0/3666884684 conn(0x7fe5640b8d10 msgr2=0x7fe56409a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:20.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 -- 172.21.15.122:0/3666884684 shutdown_connections 2024-03-16T20:45:20.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 -- 172.21.15.122:0/3666884684 wait complete. 2024-03-16T20:45:20.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 Processor -- start 2024-03-16T20:45:20.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 -- start start 2024-03-16T20:45:20.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe56406ddb0 0x7fe564147070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:20.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5640b3370 0x7fe5640b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:20.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5641475f0 0x7fe5641e2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:20.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe564146be0 con 0x7fe56406ddb0 2024-03-16T20:45:20.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe564145150 con 0x7fe5640b3370 2024-03-16T20:45:20.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5640996a0 con 0x7fe5641475f0 2024-03-16T20:45:20.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe5637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe56406ddb0 0x7fe564147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:20.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe5637fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe56406ddb0 0x7fe564147070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50888/0 (socket says 172.21.15.122:50888) 2024-03-16T20:45:20.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe5637fe640 1 -- 172.21.15.122:0/1753668103 learned_addr learned my addr 172.21.15.122:0/1753668103 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:20.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe563fff640 1 --2- 172.21.15.122:0/1753668103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5641475f0 0x7fe5641e2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:20.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe562ffd640 1 --2- 172.21.15.122:0/1753668103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5640b3370 0x7fe5640b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:20.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe5637fe640 1 -- 172.21.15.122:0/1753668103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5641475f0 msgr2=0x7fe5641e2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:20.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe5637fe640 1 --2- 172.21.15.122:0/1753668103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5641475f0 0x7fe5641e2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe5637fe640 1 -- 172.21.15.122:0/1753668103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5640b3370 msgr2=0x7fe5640b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:20.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe5637fe640 1 --2- 172.21.15.122:0/1753668103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5640b3370 0x7fe5640b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe5637fe640 1 -- 172.21.15.122:0/1753668103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55800b6b0 con 0x7fe56406ddb0 2024-03-16T20:45:20.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe562ffd640 1 --2- 172.21.15.122:0/1753668103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5640b3370 0x7fe5640b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:20.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe5637fe640 1 --2- 172.21.15.122:0/1753668103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe56406ddb0 0x7fe564147070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe5540011f0 tx=0x7fe55400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:20.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe54bfff640 1 -- 172.21.15.122:0/1753668103 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5540074e0 con 0x7fe56406ddb0 2024-03-16T20:45:20.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe54bfff640 1 -- 172.21.15.122:0/1753668103 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe554007640 con 0x7fe56406ddb0 2024-03-16T20:45:20.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 -- 172.21.15.122:0/1753668103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5641453f0 con 0x7fe56406ddb0 2024-03-16T20:45:20.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe54bfff640 1 -- 172.21.15.122:0/1753668103 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe554004a10 con 0x7fe56406ddb0 2024-03-16T20:45:20.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 -- 172.21.15.122:0/1753668103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe564143580 con 0x7fe56406ddb0 2024-03-16T20:45:20.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.804+0000 7fe56a134640 1 -- 172.21.15.122:0/1753668103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe564074bb0 con 0x7fe56406ddb0 2024-03-16T20:45:20.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.808+0000 7fe54bfff640 1 -- 172.21.15.122:0/1753668103 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe554004b70 con 0x7fe56406ddb0 2024-03-16T20:45:20.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.808+0000 7fe54bfff640 1 --2- 172.21.15.122:0/1753668103 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe550066e40 0x7fe550069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:20.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.808+0000 7fe54bfff640 1 -- 172.21.15.122:0/1753668103 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe554083f20 con 0x7fe56406ddb0 2024-03-16T20:45:20.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.808+0000 7fe562ffd640 1 --2- 172.21.15.122:0/1753668103 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe550066e40 0x7fe550069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:20.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.808+0000 7fe562ffd640 1 --2- 172.21.15.122:0/1753668103 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe550066e40 0x7fe550069300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe5580b2000 tx=0x7fe5580a9070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:20.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.812+0000 7fe54bfff640 1 -- 172.21.15.122:0/1753668103 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe554050da0 con 0x7fe56406ddb0 2024-03-16T20:45:20.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.972+0000 7fe56a134640 1 -- 172.21.15.122:0/1753668103 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe56409a890 con 0x7fe56406ddb0 2024-03-16T20:45:20.977 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.972+0000 7fe54bfff640 1 -- 172.21.15.122:0/1753668103 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe5540180a0 con 0x7fe56406ddb0 2024-03-16T20:45:20.977 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-03-16T20:45:20.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.976+0000 7fe56a134640 1 -- 172.21.15.122:0/1753668103 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe550066e40 msgr2=0x7fe550069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:20.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.976+0000 7fe56a134640 1 --2- 172.21.15.122:0/1753668103 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe550066e40 0x7fe550069300 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe5580b2000 tx=0x7fe5580a9070 comp rx=0 tx=0).stop 2024-03-16T20:45:20.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.976+0000 7fe56a134640 1 -- 172.21.15.122:0/1753668103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe56406ddb0 msgr2=0x7fe564147070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:20.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.976+0000 7fe56a134640 1 --2- 172.21.15.122:0/1753668103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe56406ddb0 0x7fe564147070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe5540011f0 tx=0x7fe55400c9b0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.976+0000 7fe56a134640 1 -- 172.21.15.122:0/1753668103 shutdown_connections 2024-03-16T20:45:20.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.976+0000 7fe56a134640 1 --2- 172.21.15.122:0/1753668103 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe550066e40 0x7fe550069300 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.976+0000 7fe56a134640 1 --2- 172.21.15.122:0/1753668103 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe5641475f0 0x7fe5641e2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.976+0000 7fe56a134640 1 --2- 172.21.15.122:0/1753668103 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe5640b3370 0x7fe5640b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.976+0000 7fe56a134640 1 --2- 172.21.15.122:0/1753668103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe56406ddb0 0x7fe564147070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:20.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.976+0000 7fe56a134640 1 -- 172.21.15.122:0/1753668103 >> 172.21.15.122:0/1753668103 conn(0x7fe5640b8d10 msgr2=0x7fe564098ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:20.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.976+0000 7fe56a134640 1 -- 172.21.15.122:0/1753668103 shutdown_connections 2024-03-16T20:45:20.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:20.976+0000 7fe56a134640 1 -- 172.21.15.122:0/1753668103 wait complete. 2024-03-16T20:45:20.996 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T20:45:21.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.108+0000 7eff09ee7640 1 Processor -- start 2024-03-16T20:45:21.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.108+0000 7eff09ee7640 1 -- start start 2024-03-16T20:45:21.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff09ee7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0409ae30 0x7eff040b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:21.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff09ee7640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7eff04144aa0 0x7eff0414e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:21.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff09ee7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0409b210 con 0x7eff040aef80 2024-03-16T20:45:21.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff09ee7640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff040b3110 con 0x7eff04144aa0 2024-03-16T20:45:21.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff09ee7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff040b3970 con 0x7eff0409ae30 2024-03-16T20:45:21.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff03fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7eff04144aa0 0x7eff0414e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:21.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff03fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7eff04144aa0 0x7eff0414e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50902/0 (socket says 172.21.15.122:50902) 2024-03-16T20:45:21.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff03fff640 1 -- 172.21.15.122:0/3422228235 learned_addr learned my addr 172.21.15.122:0/3422228235 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:21.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff037fe640 1 --2- 172.21.15.122:0/3422228235 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0409ae30 0x7eff040b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:21.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff03fff640 1 -- 172.21.15.122:0/3422228235 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0409ae30 msgr2=0x7eff040b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:21.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff03fff640 1 --2- 172.21.15.122:0/3422228235 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0409ae30 0x7eff040b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff03fff640 1 -- 172.21.15.122:0/3422228235 >> v1:172.21.15.195:6789/0 conn(0x7eff040aef80 legacy=0x7eff04144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff03fff640 1 -- 172.21.15.122:0/3422228235 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff040b3320 con 0x7eff04144aa0 2024-03-16T20:45:21.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff03fff640 1 --2- 172.21.15.122:0/3422228235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff04144aa0 0x7eff0414e850 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efef4009870 tx=0x7efef401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff39ed8e6b65ebf server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:21.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff027fc640 1 -- 172.21.15.122:0/3422228235 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef40ab600 con 0x7eff04144aa0 2024-03-16T20:45:21.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff027fc640 1 -- 172.21.15.122:0/3422228235 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef40a7050 con 0x7eff04144aa0 2024-03-16T20:45:21.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff027fc640 1 -- 172.21.15.122:0/3422228235 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef40ab980 con 0x7eff04144aa0 2024-03-16T20:45:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff09ee7640 1 -- 172.21.15.122:0/3422228235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff04144aa0 msgr2=0x7eff0414e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff09ee7640 1 --2- 172.21.15.122:0/3422228235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff04144aa0 0x7eff0414e850 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7efef4009870 tx=0x7efef401c4d0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff09ee7640 1 -- 172.21.15.122:0/3422228235 shutdown_connections 2024-03-16T20:45:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff09ee7640 1 --2- 172.21.15.122:0/3422228235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff04144aa0 0x7eff0414e850 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff09ee7640 1 --2- 172.21.15.122:0/3422228235 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff0409ae30 0x7eff040b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff09ee7640 1 -- 172.21.15.122:0/3422228235 >> 172.21.15.122:0/3422228235 conn(0x7eff040b8d10 msgr2=0x7eff0409a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff09ee7640 1 -- 172.21.15.122:0/3422228235 shutdown_connections 2024-03-16T20:45:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.112+0000 7eff09ee7640 1 -- 172.21.15.122:0/3422228235 wait complete. 2024-03-16T20:45:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff09ee7640 1 Processor -- start 2024-03-16T20:45:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff09ee7640 1 -- start start 2024-03-16T20:45:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff09ee7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0409ae30 0x7eff0413b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff09ee7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff040aef80 0x7eff041ddb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:21.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff09ee7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff04144aa0 0x7eff041de060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:21.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff09ee7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff041de5a0 con 0x7eff040aef80 2024-03-16T20:45:21.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff09ee7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff04137510 con 0x7eff0409ae30 2024-03-16T20:45:21.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff09ee7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff04138200 con 0x7eff04144aa0 2024-03-16T20:45:21.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff02ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff040aef80 0x7eff041ddb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:21.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff02ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff040aef80 0x7eff041ddb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50914/0 (socket says 172.21.15.122:50914) 2024-03-16T20:45:21.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff02ffd640 1 -- 172.21.15.122:0/1810065936 learned_addr learned my addr 172.21.15.122:0/1810065936 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:21.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff037fe640 1 --2- 172.21.15.122:0/1810065936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0409ae30 0x7eff0413b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:21.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff03fff640 1 --2- 172.21.15.122:0/1810065936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff04144aa0 0x7eff041de060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:21.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff02ffd640 1 -- 172.21.15.122:0/1810065936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff04144aa0 msgr2=0x7eff041de060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff02ffd640 1 --2- 172.21.15.122:0/1810065936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff04144aa0 0x7eff041de060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff02ffd640 1 -- 172.21.15.122:0/1810065936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0409ae30 msgr2=0x7eff0413b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff02ffd640 1 --2- 172.21.15.122:0/1810065936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0409ae30 0x7eff0413b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff02ffd640 1 -- 172.21.15.122:0/1810065936 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef40094d0 con 0x7eff040aef80 2024-03-16T20:45:21.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff037fe640 1 --2- 172.21.15.122:0/1810065936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0409ae30 0x7eff0413b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:21.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff02ffd640 1 --2- 172.21.15.122:0/1810065936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff040aef80 0x7eff041ddb20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efeec00c420 tx=0x7efeec00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:21.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7efefbfff640 1 -- 172.21.15.122:0/1810065936 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec0052d0 con 0x7eff040aef80 2024-03-16T20:45:21.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7efefbfff640 1 -- 172.21.15.122:0/1810065936 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec005430 con 0x7eff040aef80 2024-03-16T20:45:21.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff09ee7640 1 -- 172.21.15.122:0/1810065936 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff04136ee0 con 0x7eff040aef80 2024-03-16T20:45:21.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7efefbfff640 1 -- 172.21.15.122:0/1810065936 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec007260 con 0x7eff040aef80 2024-03-16T20:45:21.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff09ee7640 1 -- 172.21.15.122:0/1810065936 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff04138520 con 0x7eff040aef80 2024-03-16T20:45:21.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7eff09ee7640 1 -- 172.21.15.122:0/1810065936 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff04074bb0 con 0x7eff040aef80 2024-03-16T20:45:21.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7efefbfff640 1 -- 172.21.15.122:0/1810065936 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7efeec0073c0 con 0x7eff040aef80 2024-03-16T20:45:21.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7efefbfff640 1 --2- 172.21.15.122:0/1810065936 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efee4066e70 0x7efee4069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:21.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.116+0000 7efefbfff640 1 -- 172.21.15.122:0/1810065936 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efeec084630 con 0x7eff040aef80 2024-03-16T20:45:21.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.120+0000 7eff037fe640 1 --2- 172.21.15.122:0/1810065936 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efee4066e70 0x7efee4069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:21.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.120+0000 7eff037fe640 1 --2- 172.21.15.122:0/1810065936 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efee4066e70 0x7efee4069330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efefc0073f0 tx=0x7efefc00b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:21.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.120+0000 7efefbfff640 1 -- 172.21.15.122:0/1810065936 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7efeec051490 con 0x7eff040aef80 2024-03-16T20:45:21.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.264+0000 7eff09ee7640 1 -- 172.21.15.122:0/1810065936 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7eff0409a6d0 con 0x7eff040aef80 2024-03-16T20:45:21.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.268+0000 7efefbfff640 1 -- 172.21.15.122:0/1810065936 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7efeec020020 con 0x7eff040aef80 2024-03-16T20:45:21.270 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-03-16T20:45:21.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.268+0000 7eff09ee7640 1 -- 172.21.15.122:0/1810065936 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efee4066e70 msgr2=0x7efee4069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.268+0000 7eff09ee7640 1 --2- 172.21.15.122:0/1810065936 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efee4066e70 0x7efee4069330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efefc0073f0 tx=0x7efefc00b040 comp rx=0 tx=0).stop 2024-03-16T20:45:21.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.268+0000 7eff09ee7640 1 -- 172.21.15.122:0/1810065936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff040aef80 msgr2=0x7eff041ddb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.268+0000 7eff09ee7640 1 --2- 172.21.15.122:0/1810065936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff040aef80 0x7eff041ddb20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efeec00c420 tx=0x7efeec00c8e0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.268+0000 7eff09ee7640 1 -- 172.21.15.122:0/1810065936 shutdown_connections 2024-03-16T20:45:21.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.268+0000 7eff09ee7640 1 --2- 172.21.15.122:0/1810065936 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efee4066e70 0x7efee4069330 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.268+0000 7eff09ee7640 1 --2- 172.21.15.122:0/1810065936 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7eff04144aa0 0x7eff041de060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.268+0000 7eff09ee7640 1 --2- 172.21.15.122:0/1810065936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff040aef80 0x7eff041ddb20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.268+0000 7eff09ee7640 1 --2- 172.21.15.122:0/1810065936 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7eff0409ae30 0x7eff0413b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.268+0000 7eff09ee7640 1 -- 172.21.15.122:0/1810065936 >> 172.21.15.122:0/1810065936 conn(0x7eff040b8d10 msgr2=0x7eff040b7eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:21.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.268+0000 7eff09ee7640 1 -- 172.21.15.122:0/1810065936 shutdown_connections 2024-03-16T20:45:21.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.268+0000 7eff09ee7640 1 -- 172.21.15.122:0/1810065936 wait complete. 2024-03-16T20:45:21.287 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T20:45:21.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.392+0000 7f440613c640 1 Processor -- start 2024-03-16T20:45:21.397 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.392+0000 7f440613c640 1 -- start start 2024-03-16T20:45:21.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.396+0000 7f440613c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f44000b82b0 0x7f44000b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:21.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.396+0000 7f440613c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44001422e0 0x7f4400142770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:21.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.396+0000 7f440613c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44000b3fa0 con 0x7f440009ae30 2024-03-16T20:45:21.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.396+0000 7f440613c640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44000b50a0 con 0x7f44000b82b0 2024-03-16T20:45:21.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.396+0000 7f440613c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4400142cb0 con 0x7f44001422e0 2024-03-16T20:45:21.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.396+0000 7f43fffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44001422e0 0x7f4400142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:21.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.396+0000 7f43feffd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f44000b82b0 0x7f44000b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:21.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.396+0000 7f43ff7fe640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f440009ae30 0x7f44000b7ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.122:58370/0 (socket says 172.21.15.122:58370) 2024-03-16T20:45:21.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.396+0000 7f43ff7fe640 1 -- 172.21.15.122:0/3036490929 learned_addr learned my addr 172.21.15.122:0/3036490929 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:21.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.396+0000 7f43feffd640 1 -- 172.21.15.122:0/3036490929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44001422e0 msgr2=0x7f4400142770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:21.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.396+0000 7f43feffd640 1 --2- 172.21.15.122:0/3036490929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44001422e0 0x7f4400142770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.396+0000 7f43feffd640 1 -- 172.21.15.122:0/3036490929 >> v1:172.21.15.195:6789/0 conn(0x7f440009ae30 legacy=0x7f44000b7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.396+0000 7f43feffd640 1 -- 172.21.15.122:0/3036490929 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440009b210 con 0x7f44000b82b0 2024-03-16T20:45:21.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.396+0000 7f43feffd640 1 --2- 172.21.15.122:0/3036490929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44000b82b0 0x7f44000b4430 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f440009b400 tx=0x7f43f401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ee59511e9a5ae8c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:21.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.396+0000 7f43fe7fc640 1 -- 172.21.15.122:0/3036490929 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f40a6040 con 0x7f44000b82b0 2024-03-16T20:45:21.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.396+0000 7f43fe7fc640 1 -- 172.21.15.122:0/3036490929 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f40aa600 con 0x7f44000b82b0 2024-03-16T20:45:21.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.396+0000 7f43fe7fc640 1 -- 172.21.15.122:0/3036490929 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f40aa990 con 0x7f44000b82b0 2024-03-16T20:45:21.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 -- 172.21.15.122:0/3036490929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44000b82b0 msgr2=0x7f44000b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 --2- 172.21.15.122:0/3036490929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44000b82b0 0x7f44000b4430 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f440009b400 tx=0x7f43f401c4a0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 -- 172.21.15.122:0/3036490929 shutdown_connections 2024-03-16T20:45:21.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 --2- 172.21.15.122:0/3036490929 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44001422e0 0x7f4400142770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 --2- 172.21.15.122:0/3036490929 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44000b82b0 0x7f44000b4430 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 -- 172.21.15.122:0/3036490929 >> 172.21.15.122:0/3036490929 conn(0x7f44000b8d10 msgr2=0x7f440009a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:21.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 -- 172.21.15.122:0/3036490929 shutdown_connections 2024-03-16T20:45:21.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 -- 172.21.15.122:0/3036490929 wait complete. 2024-03-16T20:45:21.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 Processor -- start 2024-03-16T20:45:21.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 -- start start 2024-03-16T20:45:21.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f440009ae30 0x7f4400071810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:21.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44000b82b0 0x7f44000b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:21.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44001422e0 0x7f44000b1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:21.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4400071d50 con 0x7f44000b82b0 2024-03-16T20:45:21.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440013ebe0 con 0x7f440009ae30 2024-03-16T20:45:21.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4400072020 con 0x7f44001422e0 2024-03-16T20:45:21.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f43feffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44000b82b0 0x7f44000b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:21.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f43feffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44000b82b0 0x7f44000b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50930/0 (socket says 172.21.15.122:50930) 2024-03-16T20:45:21.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f43feffd640 1 -- 172.21.15.122:0/2514416146 learned_addr learned my addr 172.21.15.122:0/2514416146 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:21.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f43ff7fe640 1 --2- 172.21.15.122:0/2514416146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f440009ae30 0x7f4400071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:21.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f43fffff640 1 --2- 172.21.15.122:0/2514416146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44001422e0 0x7f44000b1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:21.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f43feffd640 1 -- 172.21.15.122:0/2514416146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44001422e0 msgr2=0x7f44000b1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.405 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f43feffd640 1 --2- 172.21.15.122:0/2514416146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44001422e0 0x7f44000b1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f43feffd640 1 -- 172.21.15.122:0/2514416146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f440009ae30 msgr2=0x7f4400071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f43feffd640 1 --2- 172.21.15.122:0/2514416146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f440009ae30 0x7f4400071810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f43feffd640 1 -- 172.21.15.122:0/2514416146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f4009420 con 0x7f44000b82b0 2024-03-16T20:45:21.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f43feffd640 1 --2- 172.21.15.122:0/2514416146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44000b82b0 0x7f44000b1950 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f43f401a0d0 tx=0x7f43f401ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:21.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f43e7fff640 1 -- 172.21.15.122:0/2514416146 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f40b3600 con 0x7f44000b82b0 2024-03-16T20:45:21.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f43e7fff640 1 -- 172.21.15.122:0/2514416146 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f40b3760 con 0x7f44000b82b0 2024-03-16T20:45:21.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 -- 172.21.15.122:0/2514416146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44000b1150 con 0x7f44000b82b0 2024-03-16T20:45:21.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f43e7fff640 1 -- 172.21.15.122:0/2514416146 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f40b3ad0 con 0x7f44000b82b0 2024-03-16T20:45:21.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 -- 172.21.15.122:0/2514416146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440013e900 con 0x7f44000b82b0 2024-03-16T20:45:21.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.400+0000 7f440613c640 1 -- 172.21.15.122:0/2514416146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4400074bb0 con 0x7f44000b82b0 2024-03-16T20:45:21.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.404+0000 7f43e7fff640 1 -- 172.21.15.122:0/2514416146 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f43f40b3c30 con 0x7f44000b82b0 2024-03-16T20:45:21.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.404+0000 7f43e7fff640 1 --2- 172.21.15.122:0/2514416146 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f43ec066e70 0x7f43ec069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:21.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.404+0000 7f43e7fff640 1 -- 172.21.15.122:0/2514416146 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43f411edc0 con 0x7f44000b82b0 2024-03-16T20:45:21.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.404+0000 7f43ff7fe640 1 --2- 172.21.15.122:0/2514416146 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f43ec066e70 0x7f43ec069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:21.407 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.404+0000 7f43ff7fe640 1 --2- 172.21.15.122:0/2514416146 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f43ec066e70 0x7f43ec069330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f43f0005890 tx=0x7f43f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:21.410 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.408+0000 7f43e7fff640 1 -- 172.21.15.122:0/2514416146 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f43f40ebcf0 con 0x7f44000b82b0 2024-03-16T20:45:21.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.556+0000 7f440613c640 1 -- 172.21.15.122:0/2514416146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4400071e90 con 0x7f44000b82b0 2024-03-16T20:45:21.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.556+0000 7f43e7fff640 1 -- 172.21.15.122:0/2514416146 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43f40ef3b0 con 0x7f44000b82b0 2024-03-16T20:45:21.559 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-03-16T20:45:21.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.556+0000 7f440613c640 1 -- 172.21.15.122:0/2514416146 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f43ec066e70 msgr2=0x7f43ec069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.556+0000 7f440613c640 1 --2- 172.21.15.122:0/2514416146 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f43ec066e70 0x7f43ec069330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f43f0005890 tx=0x7f43f0009040 comp rx=0 tx=0).stop 2024-03-16T20:45:21.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.556+0000 7f440613c640 1 -- 172.21.15.122:0/2514416146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44000b82b0 msgr2=0x7f44000b1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.556+0000 7f440613c640 1 --2- 172.21.15.122:0/2514416146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44000b82b0 0x7f44000b1950 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f43f401a0d0 tx=0x7f43f401ace0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.556+0000 7f440613c640 1 -- 172.21.15.122:0/2514416146 shutdown_connections 2024-03-16T20:45:21.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.556+0000 7f440613c640 1 --2- 172.21.15.122:0/2514416146 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f43ec066e70 0x7f43ec069330 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.556+0000 7f440613c640 1 --2- 172.21.15.122:0/2514416146 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44001422e0 0x7f44000b1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.556+0000 7f440613c640 1 --2- 172.21.15.122:0/2514416146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44000b82b0 0x7f44000b1950 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.556+0000 7f440613c640 1 --2- 172.21.15.122:0/2514416146 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f440009ae30 0x7f4400071810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.556+0000 7f440613c640 1 -- 172.21.15.122:0/2514416146 >> 172.21.15.122:0/2514416146 conn(0x7f44000b8d10 msgr2=0x7f440013d210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:21.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.556+0000 7f440613c640 1 -- 172.21.15.122:0/2514416146 shutdown_connections 2024-03-16T20:45:21.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.556+0000 7f440613c640 1 -- 172.21.15.122:0/2514416146 wait complete. 2024-03-16T20:45:21.578 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:21.693 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.688+0000 7f648d2de640 1 Processor -- start 2024-03-16T20:45:21.694 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.688+0000 7f648d2de640 1 -- start start 2024-03-16T20:45:21.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.692+0000 7f648d2de640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6488148b40 0x7f648813f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:21.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.692+0000 7f648d2de640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64880b02a0 con 0x7f648813ed60 2024-03-16T20:45:21.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.692+0000 7f648d2de640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64880b1050 con 0x7f6488072120 2024-03-16T20:45:21.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.692+0000 7f648d2de640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6488072500 con 0x7f6488148b40 2024-03-16T20:45:21.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.692+0000 7f64877fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6488148b40 0x7f648813f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:21.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.692+0000 7f64867fc640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f648813ed60 0x7f6488148430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:51728/0 (socket says 172.21.15.122:51728) 2024-03-16T20:45:21.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.692+0000 7f64867fc640 1 -- 172.21.15.122:0/3821071538 learned_addr learned my addr 172.21.15.122:0/3821071538 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:21.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.692+0000 7f64877fe640 1 -- 172.21.15.122:0/3821071538 >> v1:172.21.15.195:6789/0 conn(0x7f6488072120 legacy=0x7f64880afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.692+0000 7f64877fe640 1 -- 172.21.15.122:0/3821071538 >> v1:172.21.15.122:6789/0 conn(0x7f648813ed60 legacy=0x7f6488148430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.692+0000 7f64877fe640 1 -- 172.21.15.122:0/3821071538 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64880b38a0 con 0x7f6488148b40 2024-03-16T20:45:21.695 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.692+0000 7f6485ffb640 1 -- 172.21.15.122:0/3821071538 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1277388684 0 0) 0x7f64880b02a0 con 0x7f648813ed60 2024-03-16T20:45:21.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.692+0000 7f64877fe640 1 --2- 172.21.15.122:0/3821071538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6488148b40 0x7f648813f1c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6480009d90 tx=0x7f648001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57cf21585ff14dc0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:21.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.692+0000 7f6485ffb640 1 -- 172.21.15.122:0/3821071538 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64800a5890 con 0x7f6488148b40 2024-03-16T20:45:21.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.692+0000 7f6485ffb640 1 -- 172.21.15.122:0/3821071538 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64800a59f0 con 0x7f6488148b40 2024-03-16T20:45:21.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.692+0000 7f6485ffb640 1 -- 172.21.15.122:0/3821071538 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64800a5d80 con 0x7f6488148b40 2024-03-16T20:45:21.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f648d2de640 1 -- 172.21.15.122:0/3821071538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6488148b40 msgr2=0x7f648813f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f648d2de640 1 --2- 172.21.15.122:0/3821071538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6488148b40 0x7f648813f1c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6480009d90 tx=0x7f648001c8e0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f648d2de640 1 -- 172.21.15.122:0/3821071538 shutdown_connections 2024-03-16T20:45:21.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f648d2de640 1 --2- 172.21.15.122:0/3821071538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6488148b40 0x7f648813f1c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f648d2de640 1 -- 172.21.15.122:0/3821071538 >> 172.21.15.122:0/3821071538 conn(0x7f64880b8860 msgr2=0x7f64880b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:21.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f648d2de640 1 -- 172.21.15.122:0/3821071538 shutdown_connections 2024-03-16T20:45:21.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f648d2de640 1 -- 172.21.15.122:0/3821071538 wait complete. 2024-03-16T20:45:21.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f648d2de640 1 Processor -- start 2024-03-16T20:45:21.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f648d2de640 1 -- start start 2024-03-16T20:45:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f648d2de640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6488072120 0x7f648806ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f648d2de640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f648813ed60 0x7f6488070a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f648d2de640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64881e2ab0 0x7f64880b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f648d2de640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64880719c0 con 0x7f6488072120 2024-03-16T20:45:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f648d2de640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6488070fd0 con 0x7f64881e2ab0 2024-03-16T20:45:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f648d2de640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64880b7d30 con 0x7f648813ed60 2024-03-16T20:45:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f6486ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6488072120 0x7f648806ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f6486ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6488072120 0x7f648806ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50946/0 (socket says 172.21.15.122:50946) 2024-03-16T20:45:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f6486ffd640 1 -- 172.21.15.122:0/1145676702 learned_addr learned my addr 172.21.15.122:0/1145676702 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:21.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f64867fc640 1 --2- 172.21.15.122:0/1145676702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f648813ed60 0x7f6488070a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f64877fe640 1 --2- 172.21.15.122:0/1145676702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64881e2ab0 0x7f64880b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f6486ffd640 1 -- 172.21.15.122:0/1145676702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f648813ed60 msgr2=0x7f6488070a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f6486ffd640 1 --2- 172.21.15.122:0/1145676702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f648813ed60 0x7f6488070a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f6486ffd640 1 -- 172.21.15.122:0/1145676702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64881e2ab0 msgr2=0x7f64880b77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f6486ffd640 1 --2- 172.21.15.122:0/1145676702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64881e2ab0 0x7f64880b77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f6486ffd640 1 -- 172.21.15.122:0/1145676702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6480009a70 con 0x7f6488072120 2024-03-16T20:45:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f64877fe640 1 --2- 172.21.15.122:0/1145676702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64881e2ab0 0x7f64880b77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f6486ffd640 1 --2- 172.21.15.122:0/1145676702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6488072120 0x7f648806ff80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c500 tx=0x7f647c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f646f7fe640 1 -- 172.21.15.122:0/1145676702 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c005220 con 0x7f6488072120 2024-03-16T20:45:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f646f7fe640 1 -- 172.21.15.122:0/1145676702 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f647c005380 con 0x7f6488072120 2024-03-16T20:45:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.696+0000 7f646f7fe640 1 -- 172.21.15.122:0/1145676702 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647c007270 con 0x7f6488072120 2024-03-16T20:45:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.700+0000 7f648d2de640 1 -- 172.21.15.122:0/1145676702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6488071250 con 0x7f6488072120 2024-03-16T20:45:21.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.700+0000 7f648d2de640 1 -- 172.21.15.122:0/1145676702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648813e290 con 0x7f6488072120 2024-03-16T20:45:21.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.700+0000 7f648d2de640 1 -- 172.21.15.122:0/1145676702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f648813e7b0 con 0x7f6488072120 2024-03-16T20:45:21.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.700+0000 7f646f7fe640 1 -- 172.21.15.122:0/1145676702 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f647c0073d0 con 0x7f6488072120 2024-03-16T20:45:21.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.700+0000 7f646f7fe640 1 --2- 172.21.15.122:0/1145676702 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6474066e40 0x7f6474069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:21.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.700+0000 7f646f7fe640 1 -- 172.21.15.122:0/1145676702 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f647c083dc0 con 0x7f6488072120 2024-03-16T20:45:21.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.700+0000 7f64867fc640 1 --2- 172.21.15.122:0/1145676702 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6474066e40 0x7f6474069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:21.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.700+0000 7f64867fc640 1 --2- 172.21.15.122:0/1145676702 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6474066e40 0x7f6474069300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6488071e90 tx=0x7f6478009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:21.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.704+0000 7f646f7fe640 1 -- 172.21.15.122:0/1145676702 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f648813e7b0 con 0x7f6488072120 2024-03-16T20:45:21.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.864+0000 7f648d2de640 1 -- 172.21.15.122:0/1145676702 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6488075280 con 0x7f6488072120 2024-03-16T20:45:21.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.868+0000 7f646f7fe640 1 -- 172.21.15.122:0/1145676702 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f647c050bc0 con 0x7f6488072120 2024-03-16T20:45:21.872 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:21.872 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:06.404479+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:21.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.872+0000 7f648d2de640 1 -- 172.21.15.122:0/1145676702 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6474066e40 msgr2=0x7f6474069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.872+0000 7f648d2de640 1 --2- 172.21.15.122:0/1145676702 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6474066e40 0x7f6474069300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6488071e90 tx=0x7f6478009040 comp rx=0 tx=0).stop 2024-03-16T20:45:21.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.872+0000 7f648d2de640 1 -- 172.21.15.122:0/1145676702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6488072120 msgr2=0x7f648806ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:21.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.872+0000 7f648d2de640 1 --2- 172.21.15.122:0/1145676702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6488072120 0x7f648806ff80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f647c00c500 tx=0x7f647c00c9c0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.872+0000 7f648d2de640 1 -- 172.21.15.122:0/1145676702 shutdown_connections 2024-03-16T20:45:21.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.872+0000 7f648d2de640 1 --2- 172.21.15.122:0/1145676702 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6474066e40 0x7f6474069300 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.872+0000 7f648d2de640 1 --2- 172.21.15.122:0/1145676702 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f64881e2ab0 0x7f64880b77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.872+0000 7f648d2de640 1 --2- 172.21.15.122:0/1145676702 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f648813ed60 0x7f6488070a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.872+0000 7f648d2de640 1 --2- 172.21.15.122:0/1145676702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6488072120 0x7f648806ff80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:21.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.872+0000 7f648d2de640 1 -- 172.21.15.122:0/1145676702 >> 172.21.15.122:0/1145676702 conn(0x7f64880b8860 msgr2=0x7f6488137780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:21.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.872+0000 7f648d2de640 1 -- 172.21.15.122:0/1145676702 shutdown_connections 2024-03-16T20:45:21.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:21.872+0000 7f648d2de640 1 -- 172.21.15.122:0/1145676702 wait complete. 2024-03-16T20:45:21.891 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T20:44:52.653523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T20:45:02.758515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T20:45:03.710783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T20:45:21.891 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T20:45:22.010 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f067e14b640 1 Processor -- start 2024-03-16T20:45:22.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f067e14b640 1 -- start start 2024-03-16T20:45:22.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f067e14b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0678066fa0 0x7f067814aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f067e14b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f0678072c20 0x7f067814afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f067e14b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06780aec70 con 0x7f06780adf80 2024-03-16T20:45:22.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f067e14b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06780ae690 con 0x7f0678072c20 2024-03-16T20:45:22.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f067e14b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0678067380 con 0x7f0678066fa0 2024-03-16T20:45:22.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f0676ffd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f06780adf80 0x7f0678072510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:51742/0 (socket says 172.21.15.122:51742) 2024-03-16T20:45:22.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f0676ffd640 1 -- 172.21.15.122:0/3475280951 learned_addr learned my addr 172.21.15.122:0/3475280951 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:22.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f0677fff640 1 --2- 172.21.15.122:0/3475280951 >> v2:172.21.15.195:3300/0 conn(0x7f0678072c20 0x7f067814afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06777fe640 1 --2- 172.21.15.122:0/3475280951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0678066fa0 0x7f067814aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06767fc640 1 -- 172.21.15.122:0/3475280951 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375886081 0 0) 0x7f06780aec70 con 0x7f06780adf80 2024-03-16T20:45:22.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06767fc640 1 -- 172.21.15.122:0/3475280951 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0660003530 con 0x7f06780adf80 2024-03-16T20:45:22.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06767fc640 1 -- 172.21.15.122:0/3475280951 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3470853659 0 0) 0x7f0660003530 con 0x7f06780adf80 2024-03-16T20:45:22.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06767fc640 1 -- 172.21.15.122:0/3475280951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0678066fa0 msgr2=0x7f067814aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:22.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06767fc640 1 --2- 172.21.15.122:0/3475280951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0678066fa0 0x7f067814aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06767fc640 1 -- 172.21.15.122:0/3475280951 >> v2:172.21.15.195:3300/0 conn(0x7f0678072c20 msgr2=0x7f067814afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:22.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06767fc640 1 --2- 172.21.15.122:0/3475280951 >> v2:172.21.15.195:3300/0 conn(0x7f0678072c20 0x7f067814afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06767fc640 1 -- 172.21.15.122:0/3475280951 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06780afa90 con 0x7f06780adf80 2024-03-16T20:45:22.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06767fc640 1 -- 172.21.15.122:0/3475280951 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f066c001680 con 0x7f06780adf80 2024-03-16T20:45:22.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06767fc640 1 -- 172.21.15.122:0/3475280951 >> v1:172.21.15.122:6789/0 conn(0x7f06780adf80 legacy=0x7f0678072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06767fc640 1 --2- 172.21.15.122:0/3475280951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0678066fa0 0x7f0660003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06767fc640 1 --2- 172.21.15.122:0/3475280951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0660003f00 0x7f06600047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06767fc640 1 --2- 172.21.15.122:0/3475280951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0660004d10 0x7f0660005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06767fc640 1 -- 172.21.15.122:0/3475280951 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06780ae690 con 0x7f0660003f00 2024-03-16T20:45:22.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06767fc640 1 -- 172.21.15.122:0/3475280951 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0678067380 con 0x7f0660004d10 2024-03-16T20:45:22.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06767fc640 1 -- 172.21.15.122:0/3475280951 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f066000ba70 con 0x7f0678066fa0 2024-03-16T20:45:22.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f0676ffd640 1 --2- 172.21.15.122:0/3475280951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0660003f00 0x7f06600047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f0677fff640 1 --2- 172.21.15.122:0/3475280951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0660004d10 0x7f0660005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06777fe640 1 --2- 172.21.15.122:0/3475280951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0678066fa0 0x7f0660003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f0676ffd640 1 -- 172.21.15.122:0/3475280951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0678066fa0 msgr2=0x7f0660003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:22.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f0676ffd640 1 --2- 172.21.15.122:0/3475280951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0678066fa0 0x7f0660003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f0676ffd640 1 -- 172.21.15.122:0/3475280951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0660004d10 msgr2=0x7f0660005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:22.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f0676ffd640 1 --2- 172.21.15.122:0/3475280951 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0660004d10 0x7f0660005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f0676ffd640 1 -- 172.21.15.122:0/3475280951 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0660003530 con 0x7f0660003f00 2024-03-16T20:45:22.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.008+0000 7f06777fe640 1 --2- 172.21.15.122:0/3475280951 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0678066fa0 0x7f0660003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:22.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f0676ffd640 1 --2- 172.21.15.122:0/3475280951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0660003f00 0x7f06600047d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f066c00c290 tx=0x7f066c01cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e22db848c8aff5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:22.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f06767fc640 1 -- 172.21.15.122:0/3475280951 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c0058e0 con 0x7f0660003f00 2024-03-16T20:45:22.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f06767fc640 1 -- 172.21.15.122:0/3475280951 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c005a40 con 0x7f0660003f00 2024-03-16T20:45:22.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f06767fc640 1 -- 172.21.15.122:0/3475280951 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c005df0 con 0x7f0660003f00 2024-03-16T20:45:22.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f067e14b640 1 -- 172.21.15.122:0/3475280951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0660003f00 msgr2=0x7f06600047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f067e14b640 1 --2- 172.21.15.122:0/3475280951 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0660003f00 0x7f06600047d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f066c00c290 tx=0x7f066c01cd00 comp rx=0 tx=0).stop 2024-03-16T20:45:22.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f0677fff640 1 -- 172.21.15.122:0/3475280951 reap_dead start 2024-03-16T20:45:22.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f067e14b640 1 -- 172.21.15.122:0/3475280951 shutdown_connections 2024-03-16T20:45:22.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f067e14b640 1 -- 172.21.15.122:0/3475280951 >> 172.21.15.122:0/3475280951 conn(0x7f067806d2b0 msgr2=0x7f067806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:22.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f067e14b640 1 -- 172.21.15.122:0/3475280951 shutdown_connections 2024-03-16T20:45:22.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f067e14b640 1 -- 172.21.15.122:0/3475280951 wait complete. 2024-03-16T20:45:22.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f067e14b640 1 Processor -- start 2024-03-16T20:45:22.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f067e14b640 1 -- start start 2024-03-16T20:45:22.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f067e14b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0660003f00 0x7f06780725e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f067e14b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0678143340 0x7f06781483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f067e14b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06781e2aa0 0x7f06781e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f067e14b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0678149490 con 0x7f0660003f00 2024-03-16T20:45:22.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f067e14b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06781437c0 con 0x7f0678143340 2024-03-16T20:45:22.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f067e14b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0678071b40 con 0x7f06781e2aa0 2024-03-16T20:45:22.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f06777fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0660003f00 0x7f06780725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f06777fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0660003f00 0x7f06780725e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50966/0 (socket says 172.21.15.122:50966) 2024-03-16T20:45:22.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f06777fe640 1 -- 172.21.15.122:0/2549934632 learned_addr learned my addr 172.21.15.122:0/2549934632 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:22.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f0676ffd640 1 --2- 172.21.15.122:0/2549934632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0678143340 0x7f06781483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f0677fff640 1 --2- 172.21.15.122:0/2549934632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06781e2aa0 0x7f06781e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f06777fe640 1 -- 172.21.15.122:0/2549934632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06781e2aa0 msgr2=0x7f06781e2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f06777fe640 1 --2- 172.21.15.122:0/2549934632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06781e2aa0 0x7f06781e2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f06777fe640 1 -- 172.21.15.122:0/2549934632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0678143340 msgr2=0x7f06781483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f06777fe640 1 --2- 172.21.15.122:0/2549934632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0678143340 0x7f06781483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f06777fe640 1 -- 172.21.15.122:0/2549934632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0678071fd0 con 0x7f0660003f00 2024-03-16T20:45:22.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.012+0000 7f06777fe640 1 --2- 172.21.15.122:0/2549934632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0660003f00 0x7f06780725e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0668010900 tx=0x7f0668010dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:22.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.016+0000 7f065ffff640 1 -- 172.21.15.122:0/2549934632 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066800ec20 con 0x7f0660003f00 2024-03-16T20:45:22.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.016+0000 7f065ffff640 1 -- 172.21.15.122:0/2549934632 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066800ed80 con 0x7f0660003f00 2024-03-16T20:45:22.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.016+0000 7f067e14b640 1 -- 172.21.15.122:0/2549934632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f067814dd20 con 0x7f0660003f00 2024-03-16T20:45:22.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.016+0000 7f065ffff640 1 -- 172.21.15.122:0/2549934632 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0668004260 con 0x7f0660003f00 2024-03-16T20:45:22.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.016+0000 7f067e14b640 1 -- 172.21.15.122:0/2549934632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f067814e0b0 con 0x7f0660003f00 2024-03-16T20:45:22.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.016+0000 7f067e14b640 1 -- 172.21.15.122:0/2549934632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f067806c720 con 0x7f0660003f00 2024-03-16T20:45:22.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.016+0000 7f065ffff640 1 -- 172.21.15.122:0/2549934632 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f06680043c0 con 0x7f0660003f00 2024-03-16T20:45:22.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.016+0000 7f065ffff640 1 --2- 172.21.15.122:0/2549934632 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0664066e40 0x7f0664069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.016+0000 7f065ffff640 1 -- 172.21.15.122:0/2549934632 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06680848a0 con 0x7f0660003f00 2024-03-16T20:45:22.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.016+0000 7f0676ffd640 1 --2- 172.21.15.122:0/2549934632 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0664066e40 0x7f0664069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.016+0000 7f0676ffd640 1 --2- 172.21.15.122:0/2549934632 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0664066e40 0x7f0664069300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f066c0aa000 tx=0x7f066c001a10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:22.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.020+0000 7f065ffff640 1 -- 172.21.15.122:0/2549934632 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0668051720 con 0x7f0660003f00 2024-03-16T20:45:22.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.168+0000 7f067e14b640 1 -- 172.21.15.122:0/2549934632 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0678074bb0 con 0x7f0660003f00 2024-03-16T20:45:22.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.168+0000 7f065ffff640 1 -- 172.21.15.122:0/2549934632 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0668054de0 con 0x7f0660003f00 2024-03-16T20:45:22.174 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-03-16T20:45:22.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.172+0000 7f067e14b640 1 -- 172.21.15.122:0/2549934632 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0664066e40 msgr2=0x7f0664069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.172+0000 7f067e14b640 1 --2- 172.21.15.122:0/2549934632 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0664066e40 0x7f0664069300 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f066c0aa000 tx=0x7f066c001a10 comp rx=0 tx=0).stop 2024-03-16T20:45:22.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.172+0000 7f067e14b640 1 -- 172.21.15.122:0/2549934632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0660003f00 msgr2=0x7f06780725e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.172+0000 7f067e14b640 1 --2- 172.21.15.122:0/2549934632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0660003f00 0x7f06780725e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0668010900 tx=0x7f0668010dc0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.172+0000 7f067e14b640 1 -- 172.21.15.122:0/2549934632 shutdown_connections 2024-03-16T20:45:22.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.172+0000 7f067e14b640 1 --2- 172.21.15.122:0/2549934632 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0664066e40 0x7f0664069300 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.172+0000 7f067e14b640 1 --2- 172.21.15.122:0/2549934632 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f06781e2aa0 0x7f06781e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.172+0000 7f067e14b640 1 --2- 172.21.15.122:0/2549934632 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0678143340 0x7f06781483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.172+0000 7f067e14b640 1 --2- 172.21.15.122:0/2549934632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0660003f00 0x7f06780725e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.172+0000 7f067e14b640 1 -- 172.21.15.122:0/2549934632 >> 172.21.15.122:0/2549934632 conn(0x7f067806d2b0 msgr2=0x7f06780b73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:22.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.172+0000 7f067e14b640 1 -- 172.21.15.122:0/2549934632 shutdown_connections 2024-03-16T20:45:22.177 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.172+0000 7f067e14b640 1 -- 172.21.15.122:0/2549934632 wait complete. 2024-03-16T20:45:22.191 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T20:45:22.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cf75e640 1 Processor -- start 2024-03-16T20:45:22.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cf75e640 1 -- start start 2024-03-16T20:45:22.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cf75e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7c80b7390 0x7fe7c80b6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cf75e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe7c80b6a60 0x7fe7c81471f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cf75e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe7c80b7770 0x7fe7c8147730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cf75e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c8070ac0 con 0x7fe7c80b6a60 2024-03-16T20:45:22.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cf75e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c806de40 con 0x7fe7c80b7770 2024-03-16T20:45:22.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cf75e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c806c4d0 con 0x7fe7c80b7390 2024-03-16T20:45:22.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cccf9640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe7c80b6a60 0x7fe7c81471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cccf9640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe7c80b6a60 0x7fe7c81471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50978/0 (socket says 172.21.15.122:50978) 2024-03-16T20:45:22.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cccf9640 1 -- 172.21.15.122:0/1056117821 learned_addr learned my addr 172.21.15.122:0/1056117821 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:22.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cdcfb640 1 --2- 172.21.15.122:0/1056117821 >> v2:172.21.15.195:3300/0 conn(0x7fe7c80b7770 0x7fe7c8147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cd4fa640 1 --2- 172.21.15.122:0/1056117821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7c80b7390 0x7fe7c80b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cccf9640 1 -- 172.21.15.122:0/1056117821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7c80b7390 msgr2=0x7fe7c80b6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:22.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cccf9640 1 --2- 172.21.15.122:0/1056117821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7c80b7390 0x7fe7c80b6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cccf9640 1 -- 172.21.15.122:0/1056117821 >> v2:172.21.15.195:3300/0 conn(0x7fe7c80b7770 msgr2=0x7fe7c8147730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:22.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cccf9640 1 --2- 172.21.15.122:0/1056117821 >> v2:172.21.15.195:3300/0 conn(0x7fe7c80b7770 0x7fe7c8147730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cccf9640 1 -- 172.21.15.122:0/1056117821 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c806c0a0 con 0x7fe7c80b6a60 2024-03-16T20:45:22.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.312+0000 7fe7cdcfb640 1 --2- 172.21.15.122:0/1056117821 >> v2:172.21.15.195:3300/0 conn(0x7fe7c80b7770 0x7fe7c8147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:22.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.316+0000 7fe7cccf9640 1 --2- 172.21.15.122:0/1056117821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7c80b6a60 0x7fe7c81471f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80679e0 tx=0x7fe7b801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba6873db55a3d20 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:22.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.316+0000 7fe7bffff640 1 -- 172.21.15.122:0/1056117821 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b80ab600 con 0x7fe7c80b6a60 2024-03-16T20:45:22.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.316+0000 7fe7bffff640 1 -- 172.21.15.122:0/1056117821 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b80a7050 con 0x7fe7c80b6a60 2024-03-16T20:45:22.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.316+0000 7fe7bffff640 1 -- 172.21.15.122:0/1056117821 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b80ab980 con 0x7fe7c80b6a60 2024-03-16T20:45:22.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.316+0000 7fe7cf75e640 1 -- 172.21.15.122:0/1056117821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7c80b6a60 msgr2=0x7fe7c81471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.316+0000 7fe7cf75e640 1 --2- 172.21.15.122:0/1056117821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7c80b6a60 0x7fe7c81471f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80679e0 tx=0x7fe7b801c400 comp rx=0 tx=0).stop 2024-03-16T20:45:22.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.316+0000 7fe7cf75e640 1 -- 172.21.15.122:0/1056117821 shutdown_connections 2024-03-16T20:45:22.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.316+0000 7fe7cf75e640 1 --2- 172.21.15.122:0/1056117821 >> v2:172.21.15.195:3300/0 conn(0x7fe7c80b7770 0x7fe7c8147730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.316+0000 7fe7cf75e640 1 --2- 172.21.15.122:0/1056117821 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7c80b6a60 0x7fe7c81471f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.316+0000 7fe7cf75e640 1 --2- 172.21.15.122:0/1056117821 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7c80b7390 0x7fe7c80b6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.316+0000 7fe7cf75e640 1 -- 172.21.15.122:0/1056117821 >> 172.21.15.122:0/1056117821 conn(0x7fe7c806fc00 msgr2=0x7fe7c8070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:22.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.316+0000 7fe7cf75e640 1 -- 172.21.15.122:0/1056117821 shutdown_connections 2024-03-16T20:45:22.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.316+0000 7fe7cf75e640 1 -- 172.21.15.122:0/1056117821 wait complete. 2024-03-16T20:45:22.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.316+0000 7fe7cf75e640 1 Processor -- start 2024-03-16T20:45:22.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.316+0000 7fe7cf75e640 1 -- start start 2024-03-16T20:45:22.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cf75e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7c80b6a60 0x7fe7c8149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cf75e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7c80b7390 0x7fe7c81e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cf75e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7c80b7770 0x7fe7c81e3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cf75e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c813f550 con 0x7fe7c80b6a60 2024-03-16T20:45:22.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cf75e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c8140950 con 0x7fe7c80b7770 2024-03-16T20:45:22.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cf75e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c813f750 con 0x7fe7c80b7390 2024-03-16T20:45:22.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cd4fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7c80b6a60 0x7fe7c8149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cd4fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7c80b6a60 0x7fe7c8149b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:50990/0 (socket says 172.21.15.122:50990) 2024-03-16T20:45:22.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cd4fa640 1 -- 172.21.15.122:0/216816145 learned_addr learned my addr 172.21.15.122:0/216816145 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:22.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cdcfb640 1 --2- 172.21.15.122:0/216816145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7c80b7770 0x7fe7c81e3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cccf9640 1 --2- 172.21.15.122:0/216816145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7c80b7390 0x7fe7c81e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cd4fa640 1 -- 172.21.15.122:0/216816145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7c80b7390 msgr2=0x7fe7c81e2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cd4fa640 1 --2- 172.21.15.122:0/216816145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7c80b7390 0x7fe7c81e2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cd4fa640 1 -- 172.21.15.122:0/216816145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7c80b7770 msgr2=0x7fe7c81e3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cd4fa640 1 --2- 172.21.15.122:0/216816145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7c80b7770 0x7fe7c81e3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cd4fa640 1 -- 172.21.15.122:0/216816145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b80094d0 con 0x7fe7c80b6a60 2024-03-16T20:45:22.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cd4fa640 1 --2- 172.21.15.122:0/216816145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7c80b6a60 0x7fe7c8149b50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400d800 tx=0x7fe7c400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:22.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7bdffb640 1 -- 172.21.15.122:0/216816145 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c40096a0 con 0x7fe7c80b6a60 2024-03-16T20:45:22.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7bdffb640 1 -- 172.21.15.122:0/216816145 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c4009800 con 0x7fe7c80b6a60 2024-03-16T20:45:22.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cf75e640 1 -- 172.21.15.122:0/216816145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c8140300 con 0x7fe7c80b6a60 2024-03-16T20:45:22.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7bdffb640 1 -- 172.21.15.122:0/216816145 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4009b70 con 0x7fe7c80b6a60 2024-03-16T20:45:22.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cf75e640 1 -- 172.21.15.122:0/216816145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c813db30 con 0x7fe7c80b6a60 2024-03-16T20:45:22.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cf75e640 1 -- 172.21.15.122:0/216816145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7c8140150 con 0x7fe7c80b6a60 2024-03-16T20:45:22.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7bdffb640 1 -- 172.21.15.122:0/216816145 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe7c4009cd0 con 0x7fe7c80b6a60 2024-03-16T20:45:22.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7bdffb640 1 --2- 172.21.15.122:0/216816145 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe7ac066e40 0x7fe7ac069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7bdffb640 1 -- 172.21.15.122:0/216816145 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7c4085130 con 0x7fe7c80b6a60 2024-03-16T20:45:22.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.320+0000 7fe7cccf9640 1 --2- 172.21.15.122:0/216816145 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe7ac066e40 0x7fe7ac069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.324+0000 7fe7cccf9640 1 --2- 172.21.15.122:0/216816145 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe7ac066e40 0x7fe7ac069300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80ad000 tx=0x7fe7b8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:22.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.324+0000 7fe7bdffb640 1 -- 172.21.15.122:0/216816145 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe7c4051fb0 con 0x7fe7c80b6a60 2024-03-16T20:45:22.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.472+0000 7fe7cf75e640 1 -- 172.21.15.122:0/216816145 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe7c80b1920 con 0x7fe7c80b6a60 2024-03-16T20:45:22.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.472+0000 7fe7bdffb640 1 -- 172.21.15.122:0/216816145 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe7c4055670 con 0x7fe7c80b6a60 2024-03-16T20:45:22.475 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-03-16T20:45:22.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.472+0000 7fe7cf75e640 1 -- 172.21.15.122:0/216816145 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe7ac066e40 msgr2=0x7fe7ac069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.472+0000 7fe7cf75e640 1 --2- 172.21.15.122:0/216816145 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe7ac066e40 0x7fe7ac069300 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80ad000 tx=0x7fe7b8008040 comp rx=0 tx=0).stop 2024-03-16T20:45:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.472+0000 7fe7cf75e640 1 -- 172.21.15.122:0/216816145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7c80b6a60 msgr2=0x7fe7c8149b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.472+0000 7fe7cf75e640 1 --2- 172.21.15.122:0/216816145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7c80b6a60 0x7fe7c8149b50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400d800 tx=0x7fe7c400dcc0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.472+0000 7fe7cf75e640 1 -- 172.21.15.122:0/216816145 shutdown_connections 2024-03-16T20:45:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.472+0000 7fe7cf75e640 1 --2- 172.21.15.122:0/216816145 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe7ac066e40 0x7fe7ac069300 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.472+0000 7fe7cf75e640 1 --2- 172.21.15.122:0/216816145 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7c80b7770 0x7fe7c81e3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.472+0000 7fe7cf75e640 1 --2- 172.21.15.122:0/216816145 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7c80b7390 0x7fe7c81e2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.472+0000 7fe7cf75e640 1 --2- 172.21.15.122:0/216816145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7c80b6a60 0x7fe7c8149b50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.472+0000 7fe7cf75e640 1 -- 172.21.15.122:0/216816145 >> 172.21.15.122:0/216816145 conn(0x7fe7c806fc00 msgr2=0x7fe7c813e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.472+0000 7fe7cf75e640 1 -- 172.21.15.122:0/216816145 shutdown_connections 2024-03-16T20:45:22.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.472+0000 7fe7cf75e640 1 -- 172.21.15.122:0/216816145 wait complete. 2024-03-16T20:45:22.492 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T20:45:22.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.600+0000 7f780582d640 1 Processor -- start 2024-03-16T20:45:22.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.600+0000 7f780582d640 1 -- start start 2024-03-16T20:45:22.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.600+0000 7f780582d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f780006c300 0x7f78000b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.600+0000 7f780582d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78001479a0 0x7f7800147e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.600+0000 7f780582d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7800147510 con 0x7f7800141020 2024-03-16T20:45:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.600+0000 7f780582d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78000b5250 con 0x7f780006c300 2024-03-16T20:45:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.600+0000 7f780582d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7800147290 con 0x7f78001479a0 2024-03-16T20:45:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.600+0000 7f77feffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f780006c300 0x7f78000b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.600+0000 7f77feffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f780006c300 0x7f78000b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51006/0 (socket says 172.21.15.122:51006) 2024-03-16T20:45:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.600+0000 7f77feffd640 1 -- 172.21.15.122:0/3848605083 learned_addr learned my addr 172.21.15.122:0/3848605083 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.600+0000 7f77f7fff640 1 --2- 172.21.15.122:0/3848605083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78001479a0 0x7f7800147e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.600+0000 7f77feffd640 1 -- 172.21.15.122:0/3848605083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78001479a0 msgr2=0x7f7800147e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.600+0000 7f77feffd640 1 --2- 172.21.15.122:0/3848605083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78001479a0 0x7f7800147e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.600+0000 7f77feffd640 1 -- 172.21.15.122:0/3848605083 >> v1:172.21.15.195:6789/0 conn(0x7f7800141020 legacy=0x7f78001414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:22.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.600+0000 7f77feffd640 1 -- 172.21.15.122:0/3848605083 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f780006d5e0 con 0x7f780006c300 2024-03-16T20:45:22.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.600+0000 7f77feffd640 1 --2- 172.21.15.122:0/3848605083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f780006c300 0x7f78000b72a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009870 tx=0x7f77f001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1a83c2d1178fe8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:22.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77fe7fc640 1 -- 172.21.15.122:0/3848605083 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f00ab600 con 0x7f780006c300 2024-03-16T20:45:22.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77fe7fc640 1 -- 172.21.15.122:0/3848605083 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f00a7050 con 0x7f780006c300 2024-03-16T20:45:22.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77fe7fc640 1 -- 172.21.15.122:0/3848605083 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f00ab980 con 0x7f780006c300 2024-03-16T20:45:22.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 -- 172.21.15.122:0/3848605083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f780006c300 msgr2=0x7f78000b72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 --2- 172.21.15.122:0/3848605083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f780006c300 0x7f78000b72a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f77f0009870 tx=0x7f77f001c400 comp rx=0 tx=0).stop 2024-03-16T20:45:22.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 -- 172.21.15.122:0/3848605083 shutdown_connections 2024-03-16T20:45:22.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 --2- 172.21.15.122:0/3848605083 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78001479a0 0x7f7800147e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 --2- 172.21.15.122:0/3848605083 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f780006c300 0x7f78000b72a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 -- 172.21.15.122:0/3848605083 >> 172.21.15.122:0/3848605083 conn(0x7f780006fc00 msgr2=0x7f7800070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:22.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 -- 172.21.15.122:0/3848605083 shutdown_connections 2024-03-16T20:45:22.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 -- 172.21.15.122:0/3848605083 wait complete. 2024-03-16T20:45:22.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 Processor -- start 2024-03-16T20:45:22.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 -- start start 2024-03-16T20:45:22.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f780006c300 0x7f78000b9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7800141020 0x7f78001e25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78001479a0 0x7f7800132530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780013a550 con 0x7f7800141020 2024-03-16T20:45:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780013a7c0 con 0x7f780006c300 2024-03-16T20:45:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780014b430 con 0x7f78001479a0 2024-03-16T20:45:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77f7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7800141020 0x7f78001e25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77f7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7800141020 0x7f78001e25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51012/0 (socket says 172.21.15.122:51012) 2024-03-16T20:45:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77f7fff640 1 -- 172.21.15.122:0/2642364717 learned_addr learned my addr 172.21.15.122:0/2642364717 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77feffd640 1 --2- 172.21.15.122:0/2642364717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f780006c300 0x7f78000b9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77ff7fe640 1 --2- 172.21.15.122:0/2642364717 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78001479a0 0x7f7800132530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77f7fff640 1 -- 172.21.15.122:0/2642364717 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78001479a0 msgr2=0x7f7800132530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77f7fff640 1 --2- 172.21.15.122:0/2642364717 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78001479a0 0x7f7800132530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77f7fff640 1 -- 172.21.15.122:0/2642364717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f780006c300 msgr2=0x7f78000b9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77f7fff640 1 --2- 172.21.15.122:0/2642364717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f780006c300 0x7f78000b9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77f7fff640 1 -- 172.21.15.122:0/2642364717 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f00094d0 con 0x7f7800141020 2024-03-16T20:45:22.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77ff7fe640 1 --2- 172.21.15.122:0/2642364717 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78001479a0 0x7f7800132530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:22.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77f7fff640 1 --2- 172.21.15.122:0/2642364717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7800141020 0x7f78001e25f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f77e800d800 tx=0x7f77e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:22.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77fcff9640 1 -- 172.21.15.122:0/2642364717 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e800b6c0 con 0x7f7800141020 2024-03-16T20:45:22.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77fcff9640 1 -- 172.21.15.122:0/2642364717 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e800b820 con 0x7f7800141020 2024-03-16T20:45:22.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 -- 172.21.15.122:0/2642364717 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7800149160 con 0x7f7800141020 2024-03-16T20:45:22.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f77fcff9640 1 -- 172.21.15.122:0/2642364717 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e800bb90 con 0x7f7800141020 2024-03-16T20:45:22.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 -- 172.21.15.122:0/2642364717 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7800149720 con 0x7f7800141020 2024-03-16T20:45:22.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.604+0000 7f780582d640 1 -- 172.21.15.122:0/2642364717 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780013a970 con 0x7f7800141020 2024-03-16T20:45:22.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.608+0000 7f77fcff9640 1 -- 172.21.15.122:0/2642364717 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f77e800bcf0 con 0x7f7800141020 2024-03-16T20:45:22.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.608+0000 7f77fcff9640 1 --2- 172.21.15.122:0/2642364717 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f77e0066e40 0x7f77e0069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.608+0000 7f77fcff9640 1 -- 172.21.15.122:0/2642364717 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77e8084ee0 con 0x7f7800141020 2024-03-16T20:45:22.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.608+0000 7f77feffd640 1 --2- 172.21.15.122:0/2642364717 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f77e0066e40 0x7f77e0069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.608+0000 7f77feffd640 1 --2- 172.21.15.122:0/2642364717 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f77e0066e40 0x7f77e0069300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f77f00ad000 tx=0x7f77f00a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:22.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.612+0000 7f77fcff9640 1 -- 172.21.15.122:0/2642364717 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f780013a970 con 0x7f7800141020 2024-03-16T20:45:22.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.760+0000 7f780582d640 1 -- 172.21.15.122:0/2642364717 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f780013a930 con 0x7f7800141020 2024-03-16T20:45:22.762 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.760+0000 7f77fcff9640 1 -- 172.21.15.122:0/2642364717 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77e8055890 con 0x7f7800141020 2024-03-16T20:45:22.763 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-03-16T20:45:22.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.760+0000 7f780582d640 1 -- 172.21.15.122:0/2642364717 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f77e0066e40 msgr2=0x7f77e0069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.760+0000 7f780582d640 1 --2- 172.21.15.122:0/2642364717 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f77e0066e40 0x7f77e0069300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f77f00ad000 tx=0x7f77f00a6000 comp rx=0 tx=0).stop 2024-03-16T20:45:22.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.760+0000 7f780582d640 1 -- 172.21.15.122:0/2642364717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7800141020 msgr2=0x7f78001e25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.760+0000 7f780582d640 1 --2- 172.21.15.122:0/2642364717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7800141020 0x7f78001e25f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f77e800d800 tx=0x7f77e800dcc0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.760+0000 7f780582d640 1 -- 172.21.15.122:0/2642364717 shutdown_connections 2024-03-16T20:45:22.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.760+0000 7f780582d640 1 --2- 172.21.15.122:0/2642364717 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f77e0066e40 0x7f77e0069300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.760+0000 7f780582d640 1 --2- 172.21.15.122:0/2642364717 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f78001479a0 0x7f7800132530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.760+0000 7f780582d640 1 --2- 172.21.15.122:0/2642364717 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7800141020 0x7f78001e25f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.760+0000 7f780582d640 1 --2- 172.21.15.122:0/2642364717 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f780006c300 0x7f78000b9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.760+0000 7f780582d640 1 -- 172.21.15.122:0/2642364717 >> 172.21.15.122:0/2642364717 conn(0x7f780006fc00 msgr2=0x7f780013b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:22.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.760+0000 7f780582d640 1 -- 172.21.15.122:0/2642364717 shutdown_connections 2024-03-16T20:45:22.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.760+0000 7f780582d640 1 -- 172.21.15.122:0/2642364717 wait complete. 2024-03-16T20:45:22.780 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:22.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac920a640 1 Processor -- start 2024-03-16T20:45:22.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac920a640 1 -- start start 2024-03-16T20:45:22.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac920a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8ac40b64c0 0x7f8ac41487f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac920a640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac40b1760 con 0x7f8ac4148d30 2024-03-16T20:45:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac920a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac40b19b0 con 0x7f8ac413f3f0 2024-03-16T20:45:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac920a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac40b3140 con 0x7f8ac40b64c0 2024-03-16T20:45:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac259c640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f8ac4148d30 0x7f8ac413ece0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:51758/0 (socket says 172.21.15.122:51758) 2024-03-16T20:45:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac259c640 1 -- 172.21.15.122:0/3620627872 learned_addr learned my addr 172.21.15.122:0/3620627872 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac2d9d640 1 --2- 172.21.15.122:0/3620627872 >> v2:172.21.15.195:3300/0 conn(0x7f8ac40b64c0 0x7f8ac41487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac1d9b640 1 -- 172.21.15.122:0/3620627872 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2656451375 0 0) 0x7f8ac40b1760 con 0x7f8ac4148d30 2024-03-16T20:45:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac1d9b640 1 -- 172.21.15.122:0/3620627872 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ab0003530 con 0x7f8ac4148d30 2024-03-16T20:45:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac1d9b640 1 -- 172.21.15.122:0/3620627872 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 210532423 0 0) 0x7f8ab0003530 con 0x7f8ac4148d30 2024-03-16T20:45:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac1d9b640 1 -- 172.21.15.122:0/3620627872 >> v2:172.21.15.195:3300/0 conn(0x7f8ac40b64c0 msgr2=0x7f8ac41487f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac1d9b640 1 --2- 172.21.15.122:0/3620627872 >> v2:172.21.15.195:3300/0 conn(0x7f8ac40b64c0 0x7f8ac41487f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac1d9b640 1 -- 172.21.15.122:0/3620627872 >> v1:172.21.15.195:6789/0 conn(0x7f8ac413f3f0 legacy=0x7f8ac414f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac1d9b640 1 -- 172.21.15.122:0/3620627872 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac40b2ed0 con 0x7f8ac4148d30 2024-03-16T20:45:22.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac1d9b640 1 -- 172.21.15.122:0/3620627872 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f8aac001680 con 0x7f8ac4148d30 2024-03-16T20:45:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac1d9b640 1 -- 172.21.15.122:0/3620627872 >> v1:172.21.15.122:6789/0 conn(0x7f8ac4148d30 legacy=0x7f8ac413ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac1d9b640 1 --2- 172.21.15.122:0/3620627872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab00039d0 0x7f8ab0003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac1d9b640 1 --2- 172.21.15.122:0/3620627872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ab00047d0 0x7f8ab0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac1d9b640 1 --2- 172.21.15.122:0/3620627872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab00052b0 0x7f8ab000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac1d9b640 1 -- 172.21.15.122:0/3620627872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac40b3140 con 0x7f8ab00047d0 2024-03-16T20:45:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac1d9b640 1 -- 172.21.15.122:0/3620627872 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab000c1b0 con 0x7f8ab00039d0 2024-03-16T20:45:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac1d9b640 1 -- 172.21.15.122:0/3620627872 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab000c370 con 0x7f8ab00052b0 2024-03-16T20:45:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac259c640 1 --2- 172.21.15.122:0/3620627872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ab00047d0 0x7f8ab0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac2d9d640 1 --2- 172.21.15.122:0/3620627872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab00039d0 0x7f8ab0003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac359e640 1 --2- 172.21.15.122:0/3620627872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab00052b0 0x7f8ab000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac259c640 1 -- 172.21.15.122:0/3620627872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab00052b0 msgr2=0x7f8ab000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac259c640 1 --2- 172.21.15.122:0/3620627872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ab00052b0 0x7f8ab000b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac259c640 1 -- 172.21.15.122:0/3620627872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab00039d0 msgr2=0x7f8ab0003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac259c640 1 --2- 172.21.15.122:0/3620627872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab00039d0 0x7f8ab0003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac259c640 1 -- 172.21.15.122:0/3620627872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab0003530 con 0x7f8ab00047d0 2024-03-16T20:45:22.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.896+0000 7f8ac359e640 1 -- 172.21.15.122:0/3620627872 reap_dead start 2024-03-16T20:45:22.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac259c640 1 --2- 172.21.15.122:0/3620627872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ab00047d0 0x7f8ab0004c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c0e0 tx=0x7f8aac01cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68efdaa51270fc5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:22.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac1d9b640 1 -- 172.21.15.122:0/3620627872 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac0058e0 con 0x7f8ab00047d0 2024-03-16T20:45:22.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac1d9b640 1 -- 172.21.15.122:0/3620627872 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aac005a40 con 0x7f8ab00047d0 2024-03-16T20:45:22.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac1d9b640 1 -- 172.21.15.122:0/3620627872 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac005e70 con 0x7f8ab00047d0 2024-03-16T20:45:22.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac920a640 1 -- 172.21.15.122:0/3620627872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ab00047d0 msgr2=0x7f8ab0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac920a640 1 --2- 172.21.15.122:0/3620627872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ab00047d0 0x7f8ab0004c50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c0e0 tx=0x7f8aac01cb80 comp rx=0 tx=0).stop 2024-03-16T20:45:22.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac920a640 1 -- 172.21.15.122:0/3620627872 shutdown_connections 2024-03-16T20:45:22.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac920a640 1 --2- 172.21.15.122:0/3620627872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ab00047d0 0x7f8ab0004c50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac920a640 1 -- 172.21.15.122:0/3620627872 >> 172.21.15.122:0/3620627872 conn(0x7f8ac40b7350 msgr2=0x7f8ac40b7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:22.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac920a640 1 -- 172.21.15.122:0/3620627872 shutdown_connections 2024-03-16T20:45:22.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac920a640 1 -- 172.21.15.122:0/3620627872 wait complete. 2024-03-16T20:45:22.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac920a640 1 Processor -- start 2024-03-16T20:45:22.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac920a640 1 -- start start 2024-03-16T20:45:22.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac920a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab00047d0 0x7f8ac4147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac920a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ac4142530 0x7f8ac4141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac920a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ac4141670 0x7f8ac406c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac920a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac406db20 con 0x7f8ac4142530 2024-03-16T20:45:22.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac920a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac406e0f0 con 0x7f8ab00047d0 2024-03-16T20:45:22.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac920a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac4141af0 con 0x7f8ac4141670 2024-03-16T20:45:22.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ac4142530 0x7f8ac4141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ac4142530 0x7f8ac4141130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51026/0 (socket says 172.21.15.122:51026) 2024-03-16T20:45:22.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac259c640 1 -- 172.21.15.122:0/1034314599 learned_addr learned my addr 172.21.15.122:0/1034314599 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:22.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac2d9d640 1 --2- 172.21.15.122:0/1034314599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab00047d0 0x7f8ac4147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac359e640 1 --2- 172.21.15.122:0/1034314599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ac4141670 0x7f8ac406c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac259c640 1 -- 172.21.15.122:0/1034314599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ac4141670 msgr2=0x7f8ac406c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac259c640 1 --2- 172.21.15.122:0/1034314599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ac4141670 0x7f8ac406c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac259c640 1 -- 172.21.15.122:0/1034314599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab00047d0 msgr2=0x7f8ac4147660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:22.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac259c640 1 --2- 172.21.15.122:0/1034314599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab00047d0 0x7f8ac4147660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:22.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac259c640 1 -- 172.21.15.122:0/1034314599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac41d0750 con 0x7f8ac4142530 2024-03-16T20:45:22.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac2d9d640 1 --2- 172.21.15.122:0/1034314599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab00047d0 0x7f8ac4147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:22.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.900+0000 7f8ac259c640 1 --2- 172.21.15.122:0/1034314599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ac4142530 0x7f8ac4141130 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c0e0 tx=0x7f8ab0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:22.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.904+0000 7f8abb7fe640 1 -- 172.21.15.122:0/1034314599 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac001680 con 0x7f8ac4142530 2024-03-16T20:45:22.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.904+0000 7f8abb7fe640 1 -- 172.21.15.122:0/1034314599 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aac0047f0 con 0x7f8ac4142530 2024-03-16T20:45:22.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.904+0000 7f8ac920a640 1 -- 172.21.15.122:0/1034314599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac4140c10 con 0x7f8ac4142530 2024-03-16T20:45:22.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.904+0000 7f8abb7fe640 1 -- 172.21.15.122:0/1034314599 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac004bc0 con 0x7f8ac4142530 2024-03-16T20:45:22.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.904+0000 7f8ac920a640 1 -- 172.21.15.122:0/1034314599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac4145720 con 0x7f8ac4142530 2024-03-16T20:45:22.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.904+0000 7f8ac920a640 1 -- 172.21.15.122:0/1034314599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac406f910 con 0x7f8ac4142530 2024-03-16T20:45:22.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.904+0000 7f8abb7fe640 1 -- 172.21.15.122:0/1034314599 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f8aac004d20 con 0x7f8ac4142530 2024-03-16T20:45:22.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.904+0000 7f8abb7fe640 1 --2- 172.21.15.122:0/1034314599 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8aa4066e40 0x7f8aa4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:22.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.904+0000 7f8abb7fe640 1 -- 172.21.15.122:0/1034314599 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8aac11e5a0 con 0x7f8ac4142530 2024-03-16T20:45:22.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.904+0000 7f8ac2d9d640 1 --2- 172.21.15.122:0/1034314599 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8aa4066e40 0x7f8aa4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:22.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.904+0000 7f8ac2d9d640 1 --2- 172.21.15.122:0/1034314599 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8aa4066e40 0x7f8aa4069300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009aa0 tx=0x7f8ab4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:22.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:22.908+0000 7f8abb7fe640 1 -- 172.21.15.122:0/1034314599 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8aac0eb400 con 0x7f8ac4142530 2024-03-16T20:45:23.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.068+0000 7f8ac920a640 1 -- 172.21.15.122:0/1034314599 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ac4075280 con 0x7f8ac4142530 2024-03-16T20:45:23.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.068+0000 7f8abb7fe640 1 -- 172.21.15.122:0/1034314599 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f8aac007010 con 0x7f8ac4142530 2024-03-16T20:45:23.074 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:23.074 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:06.404479+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:23.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.072+0000 7f8ac920a640 1 -- 172.21.15.122:0/1034314599 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8aa4066e40 msgr2=0x7f8aa4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.072+0000 7f8ac920a640 1 --2- 172.21.15.122:0/1034314599 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8aa4066e40 0x7f8aa4069300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009aa0 tx=0x7f8ab4008040 comp rx=0 tx=0).stop 2024-03-16T20:45:23.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.072+0000 7f8ac920a640 1 -- 172.21.15.122:0/1034314599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ac4142530 msgr2=0x7f8ac4141130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.072+0000 7f8ac920a640 1 --2- 172.21.15.122:0/1034314599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ac4142530 0x7f8ac4141130 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c0e0 tx=0x7f8ab0005800 comp rx=0 tx=0).stop 2024-03-16T20:45:23.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.072+0000 7f8ac920a640 1 -- 172.21.15.122:0/1034314599 shutdown_connections 2024-03-16T20:45:23.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.072+0000 7f8ac920a640 1 --2- 172.21.15.122:0/1034314599 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8aa4066e40 0x7f8aa4069300 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.072+0000 7f8ac920a640 1 --2- 172.21.15.122:0/1034314599 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8ac4141670 0x7f8ac406c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.072+0000 7f8ac920a640 1 --2- 172.21.15.122:0/1034314599 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8ac4142530 0x7f8ac4141130 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.072+0000 7f8ac920a640 1 --2- 172.21.15.122:0/1034314599 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8ab00047d0 0x7f8ac4147660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.072+0000 7f8ac920a640 1 -- 172.21.15.122:0/1034314599 >> 172.21.15.122:0/1034314599 conn(0x7f8ac40b7350 msgr2=0x7f8ac40b7730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:23.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.072+0000 7f8ac920a640 1 -- 172.21.15.122:0/1034314599 shutdown_connections 2024-03-16T20:45:23.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.072+0000 7f8ac920a640 1 -- 172.21.15.122:0/1034314599 wait complete. 2024-03-16T20:45:23.093 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T20:44:52.653523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T20:45:02.758515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T20:45:03.710783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T20:45:23.093 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T20:45:23.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.208+0000 7fe4582a9640 1 Processor -- start 2024-03-16T20:45:23.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.208+0000 7fe4582a9640 1 -- start start 2024-03-16T20:45:23.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.208+0000 7fe4582a9640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe450148f40 0x7fe4500b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.208+0000 7fe4582a9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45014d700 0x7fe45014faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.208+0000 7fe4582a9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe450149320 con 0x7fe4501483a0 2024-03-16T20:45:23.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.208+0000 7fe4582a9640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4500b3d50 con 0x7fe450148f40 2024-03-16T20:45:23.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.208+0000 7fe4582a9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4500b3140 con 0x7fe45014d700 2024-03-16T20:45:23.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.208+0000 7fe456045640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe450148f40 0x7fe4500b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.208+0000 7fe456045640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe450148f40 0x7fe4500b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51038/0 (socket says 172.21.15.122:51038) 2024-03-16T20:45:23.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.208+0000 7fe456045640 1 -- 172.21.15.122:0/338882137 learned_addr learned my addr 172.21.15.122:0/338882137 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:23.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.208+0000 7fe456846640 1 --2- 172.21.15.122:0/338882137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45014d700 0x7fe45014faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.208+0000 7fe456045640 1 -- 172.21.15.122:0/338882137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45014d700 msgr2=0x7fe45014faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:23.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.208+0000 7fe456045640 1 --2- 172.21.15.122:0/338882137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45014d700 0x7fe45014faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.208+0000 7fe456045640 1 -- 172.21.15.122:0/338882137 >> v1:172.21.15.195:6789/0 conn(0x7fe4501483a0 legacy=0x7fe4500b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.208+0000 7fe456045640 1 -- 172.21.15.122:0/338882137 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4500b6970 con 0x7fe450148f40 2024-03-16T20:45:23.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.208+0000 7fe456846640 1 --2- 172.21.15.122:0/338882137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45014d700 0x7fe45014faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:23.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.212+0000 7fe456045640 1 --2- 172.21.15.122:0/338882137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe450148f40 0x7fe4500b41e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009870 tx=0x7fe44c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e7d959a740529c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:23.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.212+0000 7fe455043640 1 -- 172.21.15.122:0/338882137 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c0ab600 con 0x7fe450148f40 2024-03-16T20:45:23.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.212+0000 7fe455043640 1 -- 172.21.15.122:0/338882137 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44c0a7050 con 0x7fe450148f40 2024-03-16T20:45:23.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.212+0000 7fe455043640 1 -- 172.21.15.122:0/338882137 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c0ab980 con 0x7fe450148f40 2024-03-16T20:45:23.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.212+0000 7fe4582a9640 1 -- 172.21.15.122:0/338882137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe450148f40 msgr2=0x7fe4500b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.212+0000 7fe4582a9640 1 --2- 172.21.15.122:0/338882137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe450148f40 0x7fe4500b41e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe44c009870 tx=0x7fe44c01c4d0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.212+0000 7fe4582a9640 1 -- 172.21.15.122:0/338882137 shutdown_connections 2024-03-16T20:45:23.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.212+0000 7fe4582a9640 1 --2- 172.21.15.122:0/338882137 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45014d700 0x7fe45014faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.212+0000 7fe4582a9640 1 --2- 172.21.15.122:0/338882137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe450148f40 0x7fe4500b41e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.212+0000 7fe4582a9640 1 -- 172.21.15.122:0/338882137 >> 172.21.15.122:0/338882137 conn(0x7fe4500b7350 msgr2=0x7fe4500b7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:23.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.212+0000 7fe4582a9640 1 -- 172.21.15.122:0/338882137 shutdown_connections 2024-03-16T20:45:23.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.212+0000 7fe4582a9640 1 -- 172.21.15.122:0/338882137 wait complete. 2024-03-16T20:45:23.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.212+0000 7fe4582a9640 1 Processor -- start 2024-03-16T20:45:23.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.212+0000 7fe4582a9640 1 -- start start 2024-03-16T20:45:23.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe4582a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4501483a0 0x7fe450130510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe4582a9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe450148f40 0x7fe4501de3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe4582a9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45014d700 0x7fe4501de920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe4582a9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4501ddf90 con 0x7fe4501483a0 2024-03-16T20:45:23.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe4582a9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe450143570 con 0x7fe450148f40 2024-03-16T20:45:23.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe4582a9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4501e1ea0 con 0x7fe45014d700 2024-03-16T20:45:23.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe456045640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4501483a0 0x7fe450130510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe456045640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4501483a0 0x7fe450130510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51042/0 (socket says 172.21.15.122:51042) 2024-03-16T20:45:23.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe456045640 1 -- 172.21.15.122:0/1028818468 learned_addr learned my addr 172.21.15.122:0/1028818468 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:23.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe456846640 1 --2- 172.21.15.122:0/1028818468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45014d700 0x7fe4501de920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe455844640 1 --2- 172.21.15.122:0/1028818468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe450148f40 0x7fe4501de3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe456045640 1 -- 172.21.15.122:0/1028818468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45014d700 msgr2=0x7fe4501de920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe456045640 1 --2- 172.21.15.122:0/1028818468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45014d700 0x7fe4501de920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe456045640 1 -- 172.21.15.122:0/1028818468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe450148f40 msgr2=0x7fe4501de3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe456045640 1 --2- 172.21.15.122:0/1028818468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe450148f40 0x7fe4501de3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe456045640 1 -- 172.21.15.122:0/1028818468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c0094d0 con 0x7fe4501483a0 2024-03-16T20:45:23.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe456045640 1 --2- 172.21.15.122:0/1028818468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4501483a0 0x7fe450130510 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe44c01a100 tx=0x7fe44c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:23.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe446ffd640 1 -- 172.21.15.122:0/1028818468 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c0b6620 con 0x7fe4501483a0 2024-03-16T20:45:23.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe446ffd640 1 -- 172.21.15.122:0/1028818468 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44c0b67a0 con 0x7fe4501483a0 2024-03-16T20:45:23.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe4582a9640 1 -- 172.21.15.122:0/1028818468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4501e2180 con 0x7fe4501483a0 2024-03-16T20:45:23.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe446ffd640 1 -- 172.21.15.122:0/1028818468 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c0b6b50 con 0x7fe4501483a0 2024-03-16T20:45:23.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe4582a9640 1 -- 172.21.15.122:0/1028818468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe450145b80 con 0x7fe4501483a0 2024-03-16T20:45:23.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe4582a9640 1 -- 172.21.15.122:0/1028818468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe450074bb0 con 0x7fe4501483a0 2024-03-16T20:45:23.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe446ffd640 1 -- 172.21.15.122:0/1028818468 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe44c0b2040 con 0x7fe4501483a0 2024-03-16T20:45:23.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe446ffd640 1 --2- 172.21.15.122:0/1028818468 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe434066e40 0x7fe434069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe446ffd640 1 -- 172.21.15.122:0/1028818468 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe44c11ef00 con 0x7fe4501483a0 2024-03-16T20:45:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe455844640 1 --2- 172.21.15.122:0/1028818468 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe434066e40 0x7fe434069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.216+0000 7fe455844640 1 --2- 172.21.15.122:0/1028818468 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe434066e40 0x7fe434069300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe4501dbbc0 tx=0x7fe440009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:23.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.220+0000 7fe446ffd640 1 -- 172.21.15.122:0/1028818468 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe44c0ebd80 con 0x7fe4501483a0 2024-03-16T20:45:23.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.376+0000 7fe4582a9640 1 -- 172.21.15.122:0/1028818468 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4500b2b80 con 0x7fe4501483a0 2024-03-16T20:45:23.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.376+0000 7fe446ffd640 1 -- 172.21.15.122:0/1028818468 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe44c0ef440 con 0x7fe4501483a0 2024-03-16T20:45:23.381 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-03-16T20:45:23.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.380+0000 7fe4582a9640 1 -- 172.21.15.122:0/1028818468 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe434066e40 msgr2=0x7fe434069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.380+0000 7fe4582a9640 1 --2- 172.21.15.122:0/1028818468 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe434066e40 0x7fe434069300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe4501dbbc0 tx=0x7fe440009040 comp rx=0 tx=0).stop 2024-03-16T20:45:23.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.380+0000 7fe4582a9640 1 -- 172.21.15.122:0/1028818468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4501483a0 msgr2=0x7fe450130510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.380+0000 7fe4582a9640 1 --2- 172.21.15.122:0/1028818468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4501483a0 0x7fe450130510 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe44c01a100 tx=0x7fe44c01ad10 comp rx=0 tx=0).stop 2024-03-16T20:45:23.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.380+0000 7fe4582a9640 1 -- 172.21.15.122:0/1028818468 shutdown_connections 2024-03-16T20:45:23.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.380+0000 7fe4582a9640 1 --2- 172.21.15.122:0/1028818468 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe434066e40 0x7fe434069300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.380+0000 7fe4582a9640 1 --2- 172.21.15.122:0/1028818468 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe45014d700 0x7fe4501de920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.380+0000 7fe4582a9640 1 --2- 172.21.15.122:0/1028818468 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe450148f40 0x7fe4501de3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.380+0000 7fe4582a9640 1 --2- 172.21.15.122:0/1028818468 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe4501483a0 0x7fe450130510 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.380+0000 7fe4582a9640 1 -- 172.21.15.122:0/1028818468 >> 172.21.15.122:0/1028818468 conn(0x7fe4500b7350 msgr2=0x7fe4500b60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:23.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.380+0000 7fe4582a9640 1 -- 172.21.15.122:0/1028818468 shutdown_connections 2024-03-16T20:45:23.384 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.380+0000 7fe4582a9640 1 -- 172.21.15.122:0/1028818468 wait complete. 2024-03-16T20:45:23.398 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T20:45:23.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf70b0d640 1 Processor -- start 2024-03-16T20:45:23.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf70b0d640 1 -- start start 2024-03-16T20:45:23.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf70b0d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c1429f0 0x7fcf6c0b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf70b0d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf6c0b3fa0 con 0x7fcf6c09ae30 2024-03-16T20:45:23.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf70b0d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf6c09b210 con 0x7fcf6c0b82b0 2024-03-16T20:45:23.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf70b0d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c0b50a0 con 0x7fcf6c1429f0 2024-03-16T20:45:23.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf6a59c640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fcf6c09ae30 0x7fcf6c0b7ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:51760/0 (socket says 172.21.15.122:51760) 2024-03-16T20:45:23.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf6a59c640 1 -- 172.21.15.122:0/2476785546 learned_addr learned my addr 172.21.15.122:0/2476785546 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:23.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf6ad9d640 1 --2- 172.21.15.122:0/2476785546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c1429f0 0x7fcf6c0b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf69d9b640 1 -- 172.21.15.122:0/2476785546 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776701123 0 0) 0x7fcf6c0b3fa0 con 0x7fcf6c09ae30 2024-03-16T20:45:23.515 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf69d9b640 1 -- 172.21.15.122:0/2476785546 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf58003530 con 0x7fcf6c09ae30 2024-03-16T20:45:23.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf69d9b640 1 -- 172.21.15.122:0/2476785546 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3595490828 0 0) 0x7fcf58003530 con 0x7fcf6c09ae30 2024-03-16T20:45:23.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf69d9b640 1 -- 172.21.15.122:0/2476785546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c1429f0 msgr2=0x7fcf6c0b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:23.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf69d9b640 1 --2- 172.21.15.122:0/2476785546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c1429f0 0x7fcf6c0b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf69d9b640 1 -- 172.21.15.122:0/2476785546 >> v1:172.21.15.195:6789/0 conn(0x7fcf6c0b82b0 legacy=0x7fcf6c1422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf69d9b640 1 -- 172.21.15.122:0/2476785546 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c0b4ae0 con 0x7fcf6c09ae30 2024-03-16T20:45:23.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf69d9b640 1 -- 172.21.15.122:0/2476785546 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7fcf540016a0 con 0x7fcf6c09ae30 2024-03-16T20:45:23.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf69d9b640 1 -- 172.21.15.122:0/2476785546 >> v1:172.21.15.122:6789/0 conn(0x7fcf6c09ae30 legacy=0x7fcf6c0b7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf69d9b640 1 --2- 172.21.15.122:0/2476785546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c1429f0 0x7fcf580039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf69d9b640 1 --2- 172.21.15.122:0/2476785546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf58003ef0 0x7fcf580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf69d9b640 1 --2- 172.21.15.122:0/2476785546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf58004d10 0x7fcf58005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf69d9b640 1 -- 172.21.15.122:0/2476785546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c0b50a0 con 0x7fcf58004d10 2024-03-16T20:45:23.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf69d9b640 1 -- 172.21.15.122:0/2476785546 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5800b9f0 con 0x7fcf58003ef0 2024-03-16T20:45:23.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf69d9b640 1 -- 172.21.15.122:0/2476785546 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5800bbb0 con 0x7fcf6c1429f0 2024-03-16T20:45:23.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf6ad9d640 1 --2- 172.21.15.122:0/2476785546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf58004d10 0x7fcf58005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf63fff640 1 --2- 172.21.15.122:0/2476785546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf58003ef0 0x7fcf580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf6ad9d640 1 -- 172.21.15.122:0/2476785546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c1429f0 msgr2=0x7fcf580039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:23.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf6ad9d640 1 --2- 172.21.15.122:0/2476785546 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c1429f0 0x7fcf580039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf6ad9d640 1 -- 172.21.15.122:0/2476785546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf58003ef0 msgr2=0x7fcf580047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:23.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf6ad9d640 1 --2- 172.21.15.122:0/2476785546 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf58003ef0 0x7fcf580047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.512+0000 7fcf6ad9d640 1 -- 172.21.15.122:0/2476785546 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf58003530 con 0x7fcf58004d10 2024-03-16T20:45:23.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf6ad9d640 1 --2- 172.21.15.122:0/2476785546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf58004d10 0x7fcf58005190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009fc0 tx=0x7fcf6401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d797e4260baad29e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:23.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf69d9b640 1 -- 172.21.15.122:0/2476785546 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf640a5cd0 con 0x7fcf58004d10 2024-03-16T20:45:23.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf69d9b640 1 -- 172.21.15.122:0/2476785546 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6400f040 con 0x7fcf58004d10 2024-03-16T20:45:23.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf69d9b640 1 -- 172.21.15.122:0/2476785546 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf640ad680 con 0x7fcf58004d10 2024-03-16T20:45:23.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf70b0d640 1 -- 172.21.15.122:0/2476785546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf58004d10 msgr2=0x7fcf58005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf70b0d640 1 --2- 172.21.15.122:0/2476785546 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf58004d10 0x7fcf58005190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009fc0 tx=0x7fcf6401ccc0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf6ad9d640 1 -- 172.21.15.122:0/2476785546 reap_dead start 2024-03-16T20:45:23.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf70b0d640 1 -- 172.21.15.122:0/2476785546 shutdown_connections 2024-03-16T20:45:23.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf70b0d640 1 -- 172.21.15.122:0/2476785546 >> 172.21.15.122:0/2476785546 conn(0x7fcf6c0b8d10 msgr2=0x7fcf6c09a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:23.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf70b0d640 1 -- 172.21.15.122:0/2476785546 shutdown_connections 2024-03-16T20:45:23.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf70b0d640 1 -- 172.21.15.122:0/2476785546 wait complete. 2024-03-16T20:45:23.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf70b0d640 1 Processor -- start 2024-03-16T20:45:23.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf70b0d640 1 -- start start 2024-03-16T20:45:23.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf70b0d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf58004d10 0x7fcf6c0b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf70b0d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c1da150 0x7fcf6c1da5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf70b0d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf6c1dd2e0 0x7fcf6c1dd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf70b0d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c0b06e0 con 0x7fcf58004d10 2024-03-16T20:45:23.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf70b0d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c0b29e0 con 0x7fcf6c1dd2e0 2024-03-16T20:45:23.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf70b0d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6c0b03b0 con 0x7fcf6c1da150 2024-03-16T20:45:23.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf6a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf58004d10 0x7fcf6c0b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf6a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf58004d10 0x7fcf6c0b17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51070/0 (socket says 172.21.15.122:51070) 2024-03-16T20:45:23.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf6a59c640 1 -- 172.21.15.122:0/2489473554 learned_addr learned my addr 172.21.15.122:0/2489473554 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf6a59c640 1 -- 172.21.15.122:0/2489473554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c1da150 msgr2=0x7fcf6c1da5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf63fff640 1 --2- 172.21.15.122:0/2489473554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c1da150 0x7fcf6c1da5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf6ad9d640 1 --2- 172.21.15.122:0/2489473554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf6c1dd2e0 0x7fcf6c1dd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf6a59c640 1 --2- 172.21.15.122:0/2489473554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c1da150 0x7fcf6c1da5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf6a59c640 1 -- 172.21.15.122:0/2489473554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf6c1dd2e0 msgr2=0x7fcf6c1dd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf6a59c640 1 --2- 172.21.15.122:0/2489473554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf6c1dd2e0 0x7fcf6c1dd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf6a59c640 1 -- 172.21.15.122:0/2489473554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c1daaf0 con 0x7fcf58004d10 2024-03-16T20:45:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf6a59c640 1 --2- 172.21.15.122:0/2489473554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf58004d10 0x7fcf6c0b17d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400f400 tx=0x7fcf5400f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.516+0000 7fcf62ffd640 1 -- 172.21.15.122:0/2489473554 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54007d00 con 0x7fcf58004d10 2024-03-16T20:45:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.520+0000 7fcf62ffd640 1 -- 172.21.15.122:0/2489473554 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf54007e80 con 0x7fcf58004d10 2024-03-16T20:45:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.520+0000 7fcf70b0d640 1 -- 172.21.15.122:0/2489473554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6c09a7b0 con 0x7fcf58004d10 2024-03-16T20:45:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.520+0000 7fcf62ffd640 1 -- 172.21.15.122:0/2489473554 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5400a2f0 con 0x7fcf58004d10 2024-03-16T20:45:23.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.520+0000 7fcf70b0d640 1 -- 172.21.15.122:0/2489473554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6c0b2020 con 0x7fcf58004d10 2024-03-16T20:45:23.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.520+0000 7fcf70b0d640 1 -- 172.21.15.122:0/2489473554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf6c074bb0 con 0x7fcf58004d10 2024-03-16T20:45:23.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.520+0000 7fcf62ffd640 1 -- 172.21.15.122:0/2489473554 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fcf5400a450 con 0x7fcf58004d10 2024-03-16T20:45:23.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.520+0000 7fcf62ffd640 1 --2- 172.21.15.122:0/2489473554 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcf4c066e70 0x7fcf4c069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.520+0000 7fcf62ffd640 1 -- 172.21.15.122:0/2489473554 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf54087790 con 0x7fcf58004d10 2024-03-16T20:45:23.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.520+0000 7fcf63fff640 1 --2- 172.21.15.122:0/2489473554 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcf4c066e70 0x7fcf4c069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.520+0000 7fcf63fff640 1 --2- 172.21.15.122:0/2489473554 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcf4c066e70 0x7fcf4c069330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c09a370 tx=0x7fcf5c007b80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:23.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.524+0000 7fcf62ffd640 1 -- 172.21.15.122:0/2489473554 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fcf54058140 con 0x7fcf58004d10 2024-03-16T20:45:23.686 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.684+0000 7fcf70b0d640 1 -- 172.21.15.122:0/2489473554 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcf6c0b04f0 con 0x7fcf58004d10 2024-03-16T20:45:23.687 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.684+0000 7fcf62ffd640 1 -- 172.21.15.122:0/2489473554 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcf5400aab0 con 0x7fcf58004d10 2024-03-16T20:45:23.687 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-03-16T20:45:23.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.684+0000 7fcf70b0d640 1 -- 172.21.15.122:0/2489473554 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcf4c066e70 msgr2=0x7fcf4c069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.684+0000 7fcf70b0d640 1 --2- 172.21.15.122:0/2489473554 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcf4c066e70 0x7fcf4c069330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c09a370 tx=0x7fcf5c007b80 comp rx=0 tx=0).stop 2024-03-16T20:45:23.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.684+0000 7fcf70b0d640 1 -- 172.21.15.122:0/2489473554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf58004d10 msgr2=0x7fcf6c0b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.684+0000 7fcf70b0d640 1 --2- 172.21.15.122:0/2489473554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf58004d10 0x7fcf6c0b17d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcf5400f400 tx=0x7fcf5400f8c0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.684+0000 7fcf70b0d640 1 -- 172.21.15.122:0/2489473554 shutdown_connections 2024-03-16T20:45:23.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.684+0000 7fcf70b0d640 1 --2- 172.21.15.122:0/2489473554 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcf4c066e70 0x7fcf4c069330 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.689 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.684+0000 7fcf70b0d640 1 --2- 172.21.15.122:0/2489473554 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcf6c1dd2e0 0x7fcf6c1dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.684+0000 7fcf70b0d640 1 --2- 172.21.15.122:0/2489473554 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcf6c1da150 0x7fcf6c1da5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.684+0000 7fcf70b0d640 1 --2- 172.21.15.122:0/2489473554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcf58004d10 0x7fcf6c0b17d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.684+0000 7fcf70b0d640 1 -- 172.21.15.122:0/2489473554 >> 172.21.15.122:0/2489473554 conn(0x7fcf6c0b8d10 msgr2=0x7fcf6c1cfe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:23.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.684+0000 7fcf70b0d640 1 -- 172.21.15.122:0/2489473554 shutdown_connections 2024-03-16T20:45:23.690 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.684+0000 7fcf70b0d640 1 -- 172.21.15.122:0/2489473554 wait complete. 2024-03-16T20:45:23.706 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T20:45:23.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff93e1c2640 1 Processor -- start 2024-03-16T20:45:23.830 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff93e1c2640 1 -- start start 2024-03-16T20:45:23.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff93e1c2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9380b2ff0 0x7ff93806af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff93e1c2640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93806dff0 con 0x7ff9380b33d0 2024-03-16T20:45:23.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff93e1c2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93806f8e0 con 0x7ff93814faa0 2024-03-16T20:45:23.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff93e1c2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93806dad0 con 0x7ff9380b2ff0 2024-03-16T20:45:23.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff936ffd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7ff9380b33d0 0x7ff938147210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:51764/0 (socket says 172.21.15.122:51764) 2024-03-16T20:45:23.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff936ffd640 1 -- 172.21.15.122:0/1684421948 learned_addr learned my addr 172.21.15.122:0/1684421948 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:23.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9377fe640 1 --2- 172.21.15.122:0/1684421948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9380b2ff0 0x7ff93806af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9367fc640 1 -- 172.21.15.122:0/1684421948 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654749242 0 0) 0x7ff93806dff0 con 0x7ff9380b33d0 2024-03-16T20:45:23.832 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9367fc640 1 -- 172.21.15.122:0/1684421948 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff920003530 con 0x7ff9380b33d0 2024-03-16T20:45:23.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9367fc640 1 -- 172.21.15.122:0/1684421948 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3585291739 0 0) 0x7ff93806f8e0 con 0x7ff93814faa0 2024-03-16T20:45:23.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9367fc640 1 -- 172.21.15.122:0/1684421948 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff93806dff0 con 0x7ff93814faa0 2024-03-16T20:45:23.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9367fc640 1 -- 172.21.15.122:0/1684421948 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3519870254 0 0) 0x7ff920003530 con 0x7ff9380b33d0 2024-03-16T20:45:23.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9367fc640 1 -- 172.21.15.122:0/1684421948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9380b2ff0 msgr2=0x7ff93806af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:23.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9367fc640 1 --2- 172.21.15.122:0/1684421948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9380b2ff0 0x7ff93806af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9367fc640 1 -- 172.21.15.122:0/1684421948 >> v1:172.21.15.195:6789/0 conn(0x7ff93814faa0 legacy=0x7ff938151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9367fc640 1 -- 172.21.15.122:0/1684421948 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff938073550 con 0x7ff9380b33d0 2024-03-16T20:45:23.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9367fc640 1 -- 172.21.15.122:0/1684421948 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7ff92c001680 con 0x7ff9380b33d0 2024-03-16T20:45:23.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9367fc640 1 -- 172.21.15.122:0/1684421948 >> v1:172.21.15.122:6789/0 conn(0x7ff9380b33d0 legacy=0x7ff938147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9367fc640 1 --2- 172.21.15.122:0/1684421948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9380b2ff0 0x7ff920003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9367fc640 1 --2- 172.21.15.122:0/1684421948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9200040a0 0x7ff9200047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9367fc640 1 --2- 172.21.15.122:0/1684421948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff920004d10 0x7ff920005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9367fc640 1 -- 172.21.15.122:0/1684421948 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93806dad0 con 0x7ff9200040a0 2024-03-16T20:45:23.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9367fc640 1 -- 172.21.15.122:0/1684421948 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92000ba70 con 0x7ff920004d10 2024-03-16T20:45:23.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9367fc640 1 -- 172.21.15.122:0/1684421948 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92000bc30 con 0x7ff9380b2ff0 2024-03-16T20:45:23.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff936ffd640 1 --2- 172.21.15.122:0/1684421948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9200040a0 0x7ff9200047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff9377fe640 1 --2- 172.21.15.122:0/1684421948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9380b2ff0 0x7ff920003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff937fff640 1 --2- 172.21.15.122:0/1684421948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff920004d10 0x7ff920005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff936ffd640 1 -- 172.21.15.122:0/1684421948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9380b2ff0 msgr2=0x7ff920003b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:23.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff936ffd640 1 --2- 172.21.15.122:0/1684421948 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9380b2ff0 0x7ff920003b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.834 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff936ffd640 1 -- 172.21.15.122:0/1684421948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff920004d10 msgr2=0x7ff920005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:23.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff936ffd640 1 --2- 172.21.15.122:0/1684421948 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff920004d10 0x7ff920005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.828+0000 7ff936ffd640 1 -- 172.21.15.122:0/1684421948 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff920003530 con 0x7ff9200040a0 2024-03-16T20:45:23.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.832+0000 7ff936ffd640 1 --2- 172.21.15.122:0/1684421948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9200040a0 0x7ff9200047d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff9380b37b0 tx=0x7ff92c01cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09ce602b7107629 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:23.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.832+0000 7ff9367fc640 1 -- 172.21.15.122:0/1684421948 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0058e0 con 0x7ff9200040a0 2024-03-16T20:45:23.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.832+0000 7ff9367fc640 1 -- 172.21.15.122:0/1684421948 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92c005a40 con 0x7ff9200040a0 2024-03-16T20:45:23.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.832+0000 7ff9367fc640 1 -- 172.21.15.122:0/1684421948 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c005df0 con 0x7ff9200040a0 2024-03-16T20:45:23.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.832+0000 7ff93e1c2640 1 -- 172.21.15.122:0/1684421948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9200040a0 msgr2=0x7ff9200047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.832+0000 7ff93e1c2640 1 --2- 172.21.15.122:0/1684421948 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9200040a0 0x7ff9200047d0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff9380b37b0 tx=0x7ff92c01cba0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.832+0000 7ff937fff640 1 -- 172.21.15.122:0/1684421948 reap_dead start 2024-03-16T20:45:23.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.832+0000 7ff93e1c2640 1 -- 172.21.15.122:0/1684421948 shutdown_connections 2024-03-16T20:45:23.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.832+0000 7ff93e1c2640 1 -- 172.21.15.122:0/1684421948 >> 172.21.15.122:0/1684421948 conn(0x7ff938070d10 msgr2=0x7ff938071120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:23.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.832+0000 7ff93e1c2640 1 -- 172.21.15.122:0/1684421948 shutdown_connections 2024-03-16T20:45:23.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.832+0000 7ff93e1c2640 1 -- 172.21.15.122:0/1684421948 wait complete. 2024-03-16T20:45:23.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.832+0000 7ff93e1c2640 1 Processor -- start 2024-03-16T20:45:23.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.832+0000 7ff93e1c2640 1 -- start start 2024-03-16T20:45:23.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff93e1c2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9200040a0 0x7ff938140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff93e1c2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff938143fa0 0x7ff938141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff93e1c2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9381424a0 0x7ff9381dfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff93e1c2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff938141500 con 0x7ff9200040a0 2024-03-16T20:45:23.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff93e1c2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9381446f0 con 0x7ff938143fa0 2024-03-16T20:45:23.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff93e1c2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff938144420 con 0x7ff9381424a0 2024-03-16T20:45:23.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff9377fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9200040a0 0x7ff938140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff9377fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9200040a0 0x7ff938140fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51084/0 (socket says 172.21.15.122:51084) 2024-03-16T20:45:23.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff9377fe640 1 -- 172.21.15.122:0/2094484157 learned_addr learned my addr 172.21.15.122:0/2094484157 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:23.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff936ffd640 1 --2- 172.21.15.122:0/2094484157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff938143fa0 0x7ff938141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff937fff640 1 --2- 172.21.15.122:0/2094484157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9381424a0 0x7ff9381dfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff9377fe640 1 -- 172.21.15.122:0/2094484157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9381424a0 msgr2=0x7ff9381dfec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff9377fe640 1 --2- 172.21.15.122:0/2094484157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9381424a0 0x7ff9381dfec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff9377fe640 1 -- 172.21.15.122:0/2094484157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff938143fa0 msgr2=0x7ff938141f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:23.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff9377fe640 1 --2- 172.21.15.122:0/2094484157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff938143fa0 0x7ff938141f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:23.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff9377fe640 1 -- 172.21.15.122:0/2094484157 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9381449d0 con 0x7ff9200040a0 2024-03-16T20:45:23.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff9377fe640 1 --2- 172.21.15.122:0/2094484157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9200040a0 0x7ff938140fc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff928011610 tx=0x7ff928011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:23.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff91ffff640 1 -- 172.21.15.122:0/2094484157 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92800f8f0 con 0x7ff9200040a0 2024-03-16T20:45:23.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff91ffff640 1 -- 172.21.15.122:0/2094484157 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92800fa50 con 0x7ff9200040a0 2024-03-16T20:45:23.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff91ffff640 1 -- 172.21.15.122:0/2094484157 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92800fdc0 con 0x7ff9200040a0 2024-03-16T20:45:23.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff93e1c2640 1 -- 172.21.15.122:0/2094484157 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93813cdc0 con 0x7ff9200040a0 2024-03-16T20:45:23.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff93e1c2640 1 -- 172.21.15.122:0/2094484157 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9381436c0 con 0x7ff9200040a0 2024-03-16T20:45:23.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff93e1c2640 1 -- 172.21.15.122:0/2094484157 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff938142880 con 0x7ff9200040a0 2024-03-16T20:45:23.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff91ffff640 1 -- 172.21.15.122:0/2094484157 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7ff928015460 con 0x7ff9200040a0 2024-03-16T20:45:23.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff91ffff640 1 --2- 172.21.15.122:0/2094484157 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff924066e70 0x7ff924069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:23.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff91ffff640 1 -- 172.21.15.122:0/2094484157 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff928084db0 con 0x7ff9200040a0 2024-03-16T20:45:23.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.836+0000 7ff936ffd640 1 --2- 172.21.15.122:0/2094484157 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff924066e70 0x7ff924069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:23.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.840+0000 7ff936ffd640 1 --2- 172.21.15.122:0/2094484157 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff924066e70 0x7ff924069330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0aa000 tx=0x7ff92c001a00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:23.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:23.844+0000 7ff91ffff640 1 -- 172.21.15.122:0/2094484157 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff938142880 con 0x7ff9200040a0 2024-03-16T20:45:24.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.000+0000 7ff93e1c2640 1 -- 172.21.15.122:0/2094484157 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff938142880 con 0x7ff9200040a0 2024-03-16T20:45:24.003 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.000+0000 7ff91ffff640 1 -- 172.21.15.122:0/2094484157 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff938142880 con 0x7ff9200040a0 2024-03-16T20:45:24.004 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-03-16T20:45:24.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.000+0000 7ff93e1c2640 1 -- 172.21.15.122:0/2094484157 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff924066e70 msgr2=0x7ff924069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.004+0000 7ff93e1c2640 1 --2- 172.21.15.122:0/2094484157 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff924066e70 0x7ff924069330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff92c0aa000 tx=0x7ff92c001a00 comp rx=0 tx=0).stop 2024-03-16T20:45:24.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.004+0000 7ff93e1c2640 1 -- 172.21.15.122:0/2094484157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9200040a0 msgr2=0x7ff938140fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.004+0000 7ff93e1c2640 1 --2- 172.21.15.122:0/2094484157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9200040a0 0x7ff938140fc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff928011610 tx=0x7ff928011ad0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.004+0000 7ff93e1c2640 1 -- 172.21.15.122:0/2094484157 shutdown_connections 2024-03-16T20:45:24.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.004+0000 7ff93e1c2640 1 --2- 172.21.15.122:0/2094484157 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff924066e70 0x7ff924069330 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.004+0000 7ff93e1c2640 1 --2- 172.21.15.122:0/2094484157 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff9381424a0 0x7ff9381dfec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.004+0000 7ff93e1c2640 1 --2- 172.21.15.122:0/2094484157 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff938143fa0 0x7ff938141f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.004+0000 7ff93e1c2640 1 --2- 172.21.15.122:0/2094484157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff9200040a0 0x7ff938140fc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.006 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.004+0000 7ff93e1c2640 1 -- 172.21.15.122:0/2094484157 >> 172.21.15.122:0/2094484157 conn(0x7ff938070d10 msgr2=0x7ff93813edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:24.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.004+0000 7ff93e1c2640 1 -- 172.21.15.122:0/2094484157 shutdown_connections 2024-03-16T20:45:24.007 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.004+0000 7ff93e1c2640 1 -- 172.21.15.122:0/2094484157 wait complete. 2024-03-16T20:45:24.022 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:24.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.132+0000 7f5238692640 1 Processor -- start 2024-03-16T20:45:24.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.132+0000 7f5238692640 1 -- start start 2024-03-16T20:45:24.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f5238692640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5230072120 0x7f52300afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f5238692640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f523013ed60 0x7f523013f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f5238692640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5230148430 0x7f52301488c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f5238692640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52300b1050 con 0x7f5230072120 2024-03-16T20:45:24.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f5238692640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5230072500 con 0x7f523013ed60 2024-03-16T20:45:24.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f5238692640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52300b38a0 con 0x7f5230148430 2024-03-16T20:45:24.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f523642e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5230072120 0x7f52300afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:24.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f523642e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5230072120 0x7f52300afb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51094/0 (socket says 172.21.15.122:51094) 2024-03-16T20:45:24.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f523642e640 1 -- 172.21.15.122:0/3985358026 learned_addr learned my addr 172.21.15.122:0/3985358026 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:24.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f5235c2d640 1 --2- 172.21.15.122:0/3985358026 >> v2:172.21.15.195:3300/0 conn(0x7f523013ed60 0x7f523013f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:24.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f5236c2f640 1 --2- 172.21.15.122:0/3985358026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5230148430 0x7f52301488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:24.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f523642e640 1 -- 172.21.15.122:0/3985358026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5230148430 msgr2=0x7f52301488c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:24.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f523642e640 1 --2- 172.21.15.122:0/3985358026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5230148430 0x7f52301488c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f523642e640 1 -- 172.21.15.122:0/3985358026 >> v2:172.21.15.195:3300/0 conn(0x7f523013ed60 msgr2=0x7f523013f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:24.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f523642e640 1 --2- 172.21.15.122:0/3985358026 >> v2:172.21.15.195:3300/0 conn(0x7f523013ed60 0x7f523013f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f523642e640 1 -- 172.21.15.122:0/3985358026 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52300b3b80 con 0x7f5230072120 2024-03-16T20:45:24.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f523642e640 1 --2- 172.21.15.122:0/3985358026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5230072120 0x7f52300afb90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f523006e840 tx=0x7f522001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67fe31c52325631 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:24.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f523542c640 1 -- 172.21.15.122:0/3985358026 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52200ab600 con 0x7f5230072120 2024-03-16T20:45:24.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f523542c640 1 -- 172.21.15.122:0/3985358026 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52200a7050 con 0x7f5230072120 2024-03-16T20:45:24.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f523542c640 1 -- 172.21.15.122:0/3985358026 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52200ab980 con 0x7f5230072120 2024-03-16T20:45:24.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f5238692640 1 -- 172.21.15.122:0/3985358026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5230072120 msgr2=0x7f52300afb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f5238692640 1 --2- 172.21.15.122:0/3985358026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5230072120 0x7f52300afb90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f523006e840 tx=0x7f522001c490 comp rx=0 tx=0).stop 2024-03-16T20:45:24.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f5238692640 1 -- 172.21.15.122:0/3985358026 shutdown_connections 2024-03-16T20:45:24.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f5238692640 1 --2- 172.21.15.122:0/3985358026 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5230148430 0x7f52301488c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f5238692640 1 --2- 172.21.15.122:0/3985358026 >> v2:172.21.15.195:3300/0 conn(0x7f523013ed60 0x7f523013f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f5238692640 1 --2- 172.21.15.122:0/3985358026 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5230072120 0x7f52300afb90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f5238692640 1 -- 172.21.15.122:0/3985358026 >> 172.21.15.122:0/3985358026 conn(0x7f52300b8860 msgr2=0x7f52300b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:24.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f5238692640 1 -- 172.21.15.122:0/3985358026 shutdown_connections 2024-03-16T20:45:24.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.136+0000 7f5238692640 1 -- 172.21.15.122:0/3985358026 wait complete. 2024-03-16T20:45:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f5238692640 1 Processor -- start 2024-03-16T20:45:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f5238692640 1 -- start start 2024-03-16T20:45:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f5238692640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5230072120 0x7f52301e2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f5238692640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f523013ed60 0x7f52300b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f5238692640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5230148430 0x7f5230134440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f5238692640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523014dee0 con 0x7f5230072120 2024-03-16T20:45:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f5238692640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52301df640 con 0x7f5230148430 2024-03-16T20:45:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f5238692640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52301df920 con 0x7f523013ed60 2024-03-16T20:45:24.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f523642e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5230072120 0x7f52301e2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f523642e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5230072120 0x7f52301e2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51108/0 (socket says 172.21.15.122:51108) 2024-03-16T20:45:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f523642e640 1 -- 172.21.15.122:0/2215077090 learned_addr learned my addr 172.21.15.122:0/2215077090 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f523642e640 1 -- 172.21.15.122:0/2215077090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f523013ed60 msgr2=0x7f52300b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f523642e640 1 --2- 172.21.15.122:0/2215077090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f523013ed60 0x7f52300b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f523642e640 1 -- 172.21.15.122:0/2215077090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5230148430 msgr2=0x7f5230134440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f523642e640 1 --2- 172.21.15.122:0/2215077090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5230148430 0x7f5230134440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f523642e640 1 -- 172.21.15.122:0/2215077090 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52200094d0 con 0x7f5230072120 2024-03-16T20:45:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f523642e640 1 --2- 172.21.15.122:0/2215077090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5230072120 0x7f52301e2b00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f52200ae040 tx=0x7f5220019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f5226ffd640 1 -- 172.21.15.122:0/2215077090 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52200b4440 con 0x7f5230072120 2024-03-16T20:45:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f5226ffd640 1 -- 172.21.15.122:0/2215077090 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52200b45a0 con 0x7f5230072120 2024-03-16T20:45:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f5238692640 1 -- 172.21.15.122:0/2215077090 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52301dfc00 con 0x7f5230072120 2024-03-16T20:45:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f5226ffd640 1 -- 172.21.15.122:0/2215077090 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52200b4930 con 0x7f5230072120 2024-03-16T20:45:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f5238692640 1 -- 172.21.15.122:0/2215077090 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5230142a70 con 0x7f5230072120 2024-03-16T20:45:24.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f5238692640 1 -- 172.21.15.122:0/2215077090 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5230075280 con 0x7f5230072120 2024-03-16T20:45:24.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f5226ffd640 1 -- 172.21.15.122:0/2215077090 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f52200b2070 con 0x7f5230072120 2024-03-16T20:45:24.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f5226ffd640 1 --2- 172.21.15.122:0/2215077090 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5214066e40 0x7f5214069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.140+0000 7f5226ffd640 1 -- 172.21.15.122:0/2215077090 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f522011ec70 con 0x7f5230072120 2024-03-16T20:45:24.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.144+0000 7f5235c2d640 1 --2- 172.21.15.122:0/2215077090 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5214066e40 0x7f5214069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:24.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.144+0000 7f5235c2d640 1 --2- 172.21.15.122:0/2215077090 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5214066e40 0x7f5214069300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5230133e00 tx=0x7f522c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:24.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.148+0000 7f5226ffd640 1 -- 172.21.15.122:0/2215077090 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f52200ebd10 con 0x7f5230072120 2024-03-16T20:45:24.302 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.300+0000 7f5238692640 1 -- 172.21.15.122:0/2215077090 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52300b0d00 con 0x7f5230072120 2024-03-16T20:45:24.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.300+0000 7f5226ffd640 1 -- 172.21.15.122:0/2215077090 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f52200b0090 con 0x7f5230072120 2024-03-16T20:45:24.305 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:24.305 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:06.404479+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:24.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.304+0000 7f5238692640 1 -- 172.21.15.122:0/2215077090 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5214066e40 msgr2=0x7f5214069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.304+0000 7f5238692640 1 --2- 172.21.15.122:0/2215077090 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5214066e40 0x7f5214069300 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5230133e00 tx=0x7f522c008040 comp rx=0 tx=0).stop 2024-03-16T20:45:24.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.304+0000 7f5238692640 1 -- 172.21.15.122:0/2215077090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5230072120 msgr2=0x7f52301e2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.304+0000 7f5238692640 1 --2- 172.21.15.122:0/2215077090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5230072120 0x7f52301e2b00 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f52200ae040 tx=0x7f5220019230 comp rx=0 tx=0).stop 2024-03-16T20:45:24.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.304+0000 7f5238692640 1 -- 172.21.15.122:0/2215077090 shutdown_connections 2024-03-16T20:45:24.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.304+0000 7f5238692640 1 --2- 172.21.15.122:0/2215077090 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5214066e40 0x7f5214069300 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.304+0000 7f5238692640 1 --2- 172.21.15.122:0/2215077090 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5230148430 0x7f5230134440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.304+0000 7f5238692640 1 --2- 172.21.15.122:0/2215077090 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f523013ed60 0x7f52300b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.304+0000 7f5238692640 1 --2- 172.21.15.122:0/2215077090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5230072120 0x7f52301e2b00 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.304+0000 7f5238692640 1 -- 172.21.15.122:0/2215077090 >> 172.21.15.122:0/2215077090 conn(0x7f52300b8860 msgr2=0x7f52300b7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:24.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.304+0000 7f5238692640 1 -- 172.21.15.122:0/2215077090 shutdown_connections 2024-03-16T20:45:24.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.304+0000 7f5238692640 1 -- 172.21.15.122:0/2215077090 wait complete. 2024-03-16T20:45:24.322 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T20:44:52.653523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T20:45:02.758515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T20:45:03.710783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T20:45:24.322 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T20:45:24.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.432+0000 7f41df1af640 1 Processor -- start 2024-03-16T20:45:24.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.432+0000 7f41df1af640 1 -- start start 2024-03-16T20:45:24.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.432+0000 7f41df1af640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f41d80b3370 0x7f41d809b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.432+0000 7f41df1af640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d809df60 con 0x7f41d806ddb0 2024-03-16T20:45:24.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.432+0000 7f41df1af640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d80b3750 con 0x7f41d81475f0 2024-03-16T20:45:24.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.432+0000 7f41df1af640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d80b4a50 con 0x7f41d80b3370 2024-03-16T20:45:24.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.432+0000 7f41dcf4b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f41d80b3370 0x7f41d809b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:24.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.432+0000 7f41d7fff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f41d806ddb0 0x7f41d8146ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:51780/0 (socket says 172.21.15.122:51780) 2024-03-16T20:45:24.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.432+0000 7f41d7fff640 1 -- 172.21.15.122:0/1663646694 learned_addr learned my addr 172.21.15.122:0/1663646694 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:24.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.432+0000 7f41dcf4b640 1 --2- 172.21.15.122:0/1663646694 >> v2:172.21.15.122:3300/0 conn(0x7f41d80b3370 0x7f41d809b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51112/0 (socket says 172.21.15.122:51112) 2024-03-16T20:45:24.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.432+0000 7f41dcf4b640 1 -- 172.21.15.122:0/1663646694 >> v1:172.21.15.195:6789/0 conn(0x7f41d81475f0 legacy=0x7f41d814e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.432+0000 7f41dcf4b640 1 -- 172.21.15.122:0/1663646694 >> v1:172.21.15.122:6789/0 conn(0x7f41d806ddb0 legacy=0x7f41d8146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.432+0000 7f41dcf4b640 1 -- 172.21.15.122:0/1663646694 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d80b7fe0 con 0x7f41d80b3370 2024-03-16T20:45:24.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.432+0000 7f41dcf4b640 1 --2- 172.21.15.122:0/1663646694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41d80b3370 0x7f41d809b7c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f41c80097c0 tx=0x7f41c801c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5567371c810ae1f0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:24.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.432+0000 7f41d77fe640 1 -- 172.21.15.122:0/1663646694 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c80ab600 con 0x7f41d80b3370 2024-03-16T20:45:24.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.432+0000 7f41d77fe640 1 -- 172.21.15.122:0/1663646694 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c80a7050 con 0x7f41d80b3370 2024-03-16T20:45:24.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.432+0000 7f41d77fe640 1 -- 172.21.15.122:0/1663646694 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c80ab980 con 0x7f41d80b3370 2024-03-16T20:45:24.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41df1af640 1 -- 172.21.15.122:0/1663646694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41d80b3370 msgr2=0x7f41d809b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41df1af640 1 --2- 172.21.15.122:0/1663646694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41d80b3370 0x7f41d809b7c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f41c80097c0 tx=0x7f41c801c380 comp rx=0 tx=0).stop 2024-03-16T20:45:24.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41df1af640 1 -- 172.21.15.122:0/1663646694 shutdown_connections 2024-03-16T20:45:24.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41df1af640 1 --2- 172.21.15.122:0/1663646694 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41d80b3370 0x7f41d809b7c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41df1af640 1 -- 172.21.15.122:0/1663646694 >> 172.21.15.122:0/1663646694 conn(0x7f41d80b8d10 msgr2=0x7f41d809a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:24.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41df1af640 1 -- 172.21.15.122:0/1663646694 shutdown_connections 2024-03-16T20:45:24.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41df1af640 1 -- 172.21.15.122:0/1663646694 wait complete. 2024-03-16T20:45:24.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41df1af640 1 Processor -- start 2024-03-16T20:45:24.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41df1af640 1 -- start start 2024-03-16T20:45:24.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41df1af640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41d806ddb0 0x7f41d80702c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41df1af640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41d80b3370 0x7f41d81e1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41df1af640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41d81475f0 0x7f41d81e1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41df1af640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d81e0ea0 con 0x7f41d81475f0 2024-03-16T20:45:24.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41df1af640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d8071530 con 0x7f41d806ddb0 2024-03-16T20:45:24.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41df1af640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d813eda0 con 0x7f41d80b3370 2024-03-16T20:45:24.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41dd74c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41d81475f0 0x7f41d81e1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:24.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41dd74c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41d81475f0 0x7f41d81e1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51120/0 (socket says 172.21.15.122:51120) 2024-03-16T20:45:24.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41dd74c640 1 -- 172.21.15.122:0/1942381829 learned_addr learned my addr 172.21.15.122:0/1942381829 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:24.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41d7fff640 1 --2- 172.21.15.122:0/1942381829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41d80b3370 0x7f41d81e1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:24.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41dcf4b640 1 --2- 172.21.15.122:0/1942381829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41d806ddb0 0x7f41d80702c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:24.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41dd74c640 1 -- 172.21.15.122:0/1942381829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41d80b3370 msgr2=0x7f41d81e1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41dd74c640 1 --2- 172.21.15.122:0/1942381829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41d80b3370 0x7f41d81e1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41dd74c640 1 -- 172.21.15.122:0/1942381829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41d806ddb0 msgr2=0x7f41d80702c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41dd74c640 1 --2- 172.21.15.122:0/1942381829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41d806ddb0 0x7f41d80702c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41dd74c640 1 -- 172.21.15.122:0/1942381829 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c8009420 con 0x7f41d81475f0 2024-03-16T20:45:24.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41dd74c640 1 --2- 172.21.15.122:0/1942381829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41d81475f0 0x7f41d81e1870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f41d000c730 tx=0x7f41d000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:24.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41d57fa640 1 -- 172.21.15.122:0/1942381829 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d0003b80 con 0x7f41d81475f0 2024-03-16T20:45:24.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41d57fa640 1 -- 172.21.15.122:0/1942381829 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d0003ce0 con 0x7f41d81475f0 2024-03-16T20:45:24.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41d57fa640 1 -- 172.21.15.122:0/1942381829 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d000aa30 con 0x7f41d81475f0 2024-03-16T20:45:24.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41df1af640 1 -- 172.21.15.122:0/1942381829 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d813f080 con 0x7f41d81475f0 2024-03-16T20:45:24.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.436+0000 7f41df1af640 1 -- 172.21.15.122:0/1942381829 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d81473a0 con 0x7f41d81475f0 2024-03-16T20:45:24.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.440+0000 7f41df1af640 1 -- 172.21.15.122:0/1942381829 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d81e1050 con 0x7f41d81475f0 2024-03-16T20:45:24.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.440+0000 7f41d57fa640 1 -- 172.21.15.122:0/1942381829 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f41d000ab90 con 0x7f41d81475f0 2024-03-16T20:45:24.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.440+0000 7f41d57fa640 1 --2- 172.21.15.122:0/1942381829 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f41c4066e40 0x7f41c4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.440+0000 7f41d57fa640 1 -- 172.21.15.122:0/1942381829 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41d0084760 con 0x7f41d81475f0 2024-03-16T20:45:24.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.440+0000 7f41dcf4b640 1 --2- 172.21.15.122:0/1942381829 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f41c4066e40 0x7f41c4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:24.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.440+0000 7f41dcf4b640 1 --2- 172.21.15.122:0/1942381829 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f41c4066e40 0x7f41c4069300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f41c80ad000 tx=0x7f41c8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:24.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.444+0000 7f41d57fa640 1 -- 172.21.15.122:0/1942381829 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f41d81e1050 con 0x7f41d81475f0 2024-03-16T20:45:24.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.592+0000 7f41df1af640 1 -- 172.21.15.122:0/1942381829 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41d8074bb0 con 0x7f41d81475f0 2024-03-16T20:45:24.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.592+0000 7f41d57fa640 1 -- 172.21.15.122:0/1942381829 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41d0055110 con 0x7f41d81475f0 2024-03-16T20:45:24.595 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-03-16T20:45:24.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.592+0000 7f41df1af640 1 -- 172.21.15.122:0/1942381829 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f41c4066e40 msgr2=0x7f41c4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.592+0000 7f41df1af640 1 --2- 172.21.15.122:0/1942381829 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f41c4066e40 0x7f41c4069300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f41c80ad000 tx=0x7f41c8008040 comp rx=0 tx=0).stop 2024-03-16T20:45:24.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.592+0000 7f41df1af640 1 -- 172.21.15.122:0/1942381829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41d81475f0 msgr2=0x7f41d81e1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.592+0000 7f41df1af640 1 --2- 172.21.15.122:0/1942381829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41d81475f0 0x7f41d81e1870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f41d000c730 tx=0x7f41d000cbf0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.596+0000 7f41df1af640 1 -- 172.21.15.122:0/1942381829 shutdown_connections 2024-03-16T20:45:24.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.596+0000 7f41df1af640 1 --2- 172.21.15.122:0/1942381829 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f41c4066e40 0x7f41c4069300 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.596+0000 7f41df1af640 1 --2- 172.21.15.122:0/1942381829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41d81475f0 0x7f41d81e1870 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.596+0000 7f41df1af640 1 --2- 172.21.15.122:0/1942381829 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41d80b3370 0x7f41d81e1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.596+0000 7f41df1af640 1 --2- 172.21.15.122:0/1942381829 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41d806ddb0 0x7f41d80702c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.596+0000 7f41df1af640 1 -- 172.21.15.122:0/1942381829 >> 172.21.15.122:0/1942381829 conn(0x7f41d80b8d10 msgr2=0x7f41d8098330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:24.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.596+0000 7f41df1af640 1 -- 172.21.15.122:0/1942381829 shutdown_connections 2024-03-16T20:45:24.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.596+0000 7f41df1af640 1 -- 172.21.15.122:0/1942381829 wait complete. 2024-03-16T20:45:24.613 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T20:45:24.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef5ea15640 1 Processor -- start 2024-03-16T20:45:24.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef5ea15640 1 -- start start 2024-03-16T20:45:24.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef5ea15640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fef58148f40 0x7fef580b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef5ea15640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef5814d700 0x7fef5814faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef5ea15640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef58149320 con 0x7fef581483a0 2024-03-16T20:45:24.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef5ea15640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef580b3d50 con 0x7fef58148f40 2024-03-16T20:45:24.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef5ea15640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef580b3140 con 0x7fef5814d700 2024-03-16T20:45:24.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef57fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fef58148f40 0x7fef580b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:24.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef577fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fef581483a0 0x7fef580b6260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:51790/0 (socket says 172.21.15.122:51790) 2024-03-16T20:45:24.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef57fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fef58148f40 0x7fef580b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51122/0 (socket says 172.21.15.122:51122) 2024-03-16T20:45:24.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef577fe640 1 -- 172.21.15.122:0/1337995164 learned_addr learned my addr 172.21.15.122:0/1337995164 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:24.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef5cfb2640 1 --2- 172.21.15.122:0/1337995164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef5814d700 0x7fef5814faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:24.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef57fff640 1 -- 172.21.15.122:0/1337995164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef5814d700 msgr2=0x7fef5814faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:24.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef57fff640 1 --2- 172.21.15.122:0/1337995164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef5814d700 0x7fef5814faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef57fff640 1 -- 172.21.15.122:0/1337995164 >> v1:172.21.15.122:6789/0 conn(0x7fef581483a0 legacy=0x7fef580b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef57fff640 1 -- 172.21.15.122:0/1337995164 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef580b6970 con 0x7fef58148f40 2024-03-16T20:45:24.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef57fff640 1 --2- 172.21.15.122:0/1337995164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef58148f40 0x7fef580b41e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef48009d90 tx=0x7fef4801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1eceb11eac99106 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:24.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef56ffd640 1 -- 172.21.15.122:0/1337995164 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef480a57a0 con 0x7fef58148f40 2024-03-16T20:45:24.729 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef56ffd640 1 -- 172.21.15.122:0/1337995164 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef480a5900 con 0x7fef58148f40 2024-03-16T20:45:24.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.724+0000 7fef56ffd640 1 -- 172.21.15.122:0/1337995164 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef480a5c90 con 0x7fef58148f40 2024-03-16T20:45:24.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 -- 172.21.15.122:0/1337995164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef58148f40 msgr2=0x7fef580b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 --2- 172.21.15.122:0/1337995164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef58148f40 0x7fef580b41e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef48009d90 tx=0x7fef4801c8e0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 -- 172.21.15.122:0/1337995164 shutdown_connections 2024-03-16T20:45:24.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 --2- 172.21.15.122:0/1337995164 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef5814d700 0x7fef5814faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 --2- 172.21.15.122:0/1337995164 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef58148f40 0x7fef580b41e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 -- 172.21.15.122:0/1337995164 >> 172.21.15.122:0/1337995164 conn(0x7fef580b7350 msgr2=0x7fef580b7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:24.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 -- 172.21.15.122:0/1337995164 shutdown_connections 2024-03-16T20:45:24.731 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 -- 172.21.15.122:0/1337995164 wait complete. 2024-03-16T20:45:24.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 Processor -- start 2024-03-16T20:45:24.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 -- start start 2024-03-16T20:45:24.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef581483a0 0x7fef58138260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef58148f40 0x7fef581e0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef5814d700 0x7fef58132910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef5814ef80 con 0x7fef581483a0 2024-03-16T20:45:24.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef58145f90 con 0x7fef58148f40 2024-03-16T20:45:24.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef5814e930 con 0x7fef5814d700 2024-03-16T20:45:24.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef57fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef581483a0 0x7fef58138260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:24.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef57fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef581483a0 0x7fef58138260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51130/0 (socket says 172.21.15.122:51130) 2024-03-16T20:45:24.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef57fff640 1 -- 172.21.15.122:0/3699685602 learned_addr learned my addr 172.21.15.122:0/3699685602 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:24.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef577fe640 1 --2- 172.21.15.122:0/3699685602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef58148f40 0x7fef581e0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:24.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5cfb2640 1 --2- 172.21.15.122:0/3699685602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef5814d700 0x7fef58132910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:24.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef57fff640 1 -- 172.21.15.122:0/3699685602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef5814d700 msgr2=0x7fef58132910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef57fff640 1 --2- 172.21.15.122:0/3699685602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef5814d700 0x7fef58132910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef57fff640 1 -- 172.21.15.122:0/3699685602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef58148f40 msgr2=0x7fef581e0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef57fff640 1 --2- 172.21.15.122:0/3699685602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef58148f40 0x7fef581e0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef57fff640 1 -- 172.21.15.122:0/3699685602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef48009a70 con 0x7fef581483a0 2024-03-16T20:45:24.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef577fe640 1 --2- 172.21.15.122:0/3699685602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef58148f40 0x7fef581e0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:24.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef57fff640 1 --2- 172.21.15.122:0/3699685602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef581483a0 0x7fef58138260 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fef480b0040 tx=0x7fef48019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:24.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef54ff9640 1 -- 172.21.15.122:0/3699685602 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef480ad850 con 0x7fef581483a0 2024-03-16T20:45:24.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef54ff9640 1 -- 172.21.15.122:0/3699685602 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef480ad9b0 con 0x7fef581483a0 2024-03-16T20:45:24.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 -- 172.21.15.122:0/3699685602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5814ec10 con 0x7fef581483a0 2024-03-16T20:45:24.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef54ff9640 1 -- 172.21.15.122:0/3699685602 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef480add60 con 0x7fef581483a0 2024-03-16T20:45:24.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 -- 172.21.15.122:0/3699685602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5813f370 con 0x7fef581483a0 2024-03-16T20:45:24.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.728+0000 7fef5ea15640 1 -- 172.21.15.122:0/3699685602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef5814e790 con 0x7fef581483a0 2024-03-16T20:45:24.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.732+0000 7fef54ff9640 1 -- 172.21.15.122:0/3699685602 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fef480b4050 con 0x7fef581483a0 2024-03-16T20:45:24.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.732+0000 7fef54ff9640 1 --2- 172.21.15.122:0/3699685602 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fef44066e70 0x7fef44069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:24.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.732+0000 7fef54ff9640 1 -- 172.21.15.122:0/3699685602 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef4811f830 con 0x7fef581483a0 2024-03-16T20:45:24.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.732+0000 7fef577fe640 1 --2- 172.21.15.122:0/3699685602 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fef44066e70 0x7fef44069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:24.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.732+0000 7fef577fe640 1 --2- 172.21.15.122:0/3699685602 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fef44066e70 0x7fef44069330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fef4c007fd0 tx=0x7fef4c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:24.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.736+0000 7fef54ff9640 1 -- 172.21.15.122:0/3699685602 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fef5814e790 con 0x7fef581483a0 2024-03-16T20:45:24.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.884+0000 7fef5ea15640 1 -- 172.21.15.122:0/3699685602 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fef5814e790 con 0x7fef581483a0 2024-03-16T20:45:24.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.884+0000 7fef54ff9640 1 -- 172.21.15.122:0/3699685602 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fef5814e790 con 0x7fef581483a0 2024-03-16T20:45:24.887 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-03-16T20:45:24.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.884+0000 7fef5ea15640 1 -- 172.21.15.122:0/3699685602 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fef44066e70 msgr2=0x7fef44069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.884+0000 7fef5ea15640 1 --2- 172.21.15.122:0/3699685602 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fef44066e70 0x7fef44069330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fef4c007fd0 tx=0x7fef4c009040 comp rx=0 tx=0).stop 2024-03-16T20:45:24.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.884+0000 7fef5ea15640 1 -- 172.21.15.122:0/3699685602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef581483a0 msgr2=0x7fef58138260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:24.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.884+0000 7fef5ea15640 1 --2- 172.21.15.122:0/3699685602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef581483a0 0x7fef58138260 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fef480b0040 tx=0x7fef48019680 comp rx=0 tx=0).stop 2024-03-16T20:45:24.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.884+0000 7fef5ea15640 1 -- 172.21.15.122:0/3699685602 shutdown_connections 2024-03-16T20:45:24.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.884+0000 7fef5ea15640 1 --2- 172.21.15.122:0/3699685602 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fef44066e70 0x7fef44069330 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.884+0000 7fef5ea15640 1 --2- 172.21.15.122:0/3699685602 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fef5814d700 0x7fef58132910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.884+0000 7fef5ea15640 1 --2- 172.21.15.122:0/3699685602 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fef58148f40 0x7fef581e0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.884+0000 7fef5ea15640 1 --2- 172.21.15.122:0/3699685602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fef581483a0 0x7fef58138260 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:24.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.884+0000 7fef5ea15640 1 -- 172.21.15.122:0/3699685602 >> 172.21.15.122:0/3699685602 conn(0x7fef580b7350 msgr2=0x7fef580b60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:24.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.884+0000 7fef5ea15640 1 -- 172.21.15.122:0/3699685602 shutdown_connections 2024-03-16T20:45:24.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:24.884+0000 7fef5ea15640 1 -- 172.21.15.122:0/3699685602 wait complete. 2024-03-16T20:45:24.905 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T20:45:25.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.016+0000 7f76010d3640 1 Processor -- start 2024-03-16T20:45:25.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.016+0000 7f76010d3640 1 -- start start 2024-03-16T20:45:25.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f76010d3640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f75fc09abd0 0x7f75fc0b7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f76010d3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75fc09afb0 0x7f75fc06d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f76010d3640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75fc0b3e30 con 0x7f75fc09dd20 2024-03-16T20:45:25.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f76010d3640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75fc0b45a0 con 0x7f75fc09abd0 2024-03-16T20:45:25.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f76010d3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc0b7e60 con 0x7f75fc09afb0 2024-03-16T20:45:25.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75fa59c640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f75fc09dd20 0x7f75fc06c9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:51798/0 (socket says 172.21.15.122:51798) 2024-03-16T20:45:25.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75fa59c640 1 -- 172.21.15.122:0/3509436159 learned_addr learned my addr 172.21.15.122:0/3509436159 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:25.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75fad9d640 1 --2- 172.21.15.122:0/3509436159 >> v2:172.21.15.195:3300/0 conn(0x7f75fc09abd0 0x7f75fc0b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75fb59e640 1 --2- 172.21.15.122:0/3509436159 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75fc09afb0 0x7f75fc06d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 -- 172.21.15.122:0/3509436159 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628508008 0 0) 0x7f75fc0b3e30 con 0x7f75fc09dd20 2024-03-16T20:45:25.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 -- 172.21.15.122:0/3509436159 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75e4003530 con 0x7f75fc09dd20 2024-03-16T20:45:25.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 -- 172.21.15.122:0/3509436159 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1780232286 0 0) 0x7f75e4003530 con 0x7f75fc09dd20 2024-03-16T20:45:25.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 -- 172.21.15.122:0/3509436159 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75fc09afb0 msgr2=0x7f75fc06d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:25.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 --2- 172.21.15.122:0/3509436159 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75fc09afb0 0x7f75fc06d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 -- 172.21.15.122:0/3509436159 >> v2:172.21.15.195:3300/0 conn(0x7f75fc09abd0 msgr2=0x7f75fc0b7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:25.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 --2- 172.21.15.122:0/3509436159 >> v2:172.21.15.195:3300/0 conn(0x7f75fc09abd0 0x7f75fc0b7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 -- 172.21.15.122:0/3509436159 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75fc0b4110 con 0x7f75fc09dd20 2024-03-16T20:45:25.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 -- 172.21.15.122:0/3509436159 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f75ec0016a0 con 0x7f75fc09dd20 2024-03-16T20:45:25.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 -- 172.21.15.122:0/3509436159 >> v1:172.21.15.122:6789/0 conn(0x7f75fc09dd20 legacy=0x7f75fc06c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 --2- 172.21.15.122:0/3509436159 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75fc09afb0 0x7f75e4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 --2- 172.21.15.122:0/3509436159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75e4003f00 0x7f75e40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 --2- 172.21.15.122:0/3509436159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75e4004d10 0x7f75e4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 -- 172.21.15.122:0/3509436159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc0b45a0 con 0x7f75e4004d10 2024-03-16T20:45:25.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 -- 172.21.15.122:0/3509436159 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc0b7e60 con 0x7f75e4003f00 2024-03-16T20:45:25.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 -- 172.21.15.122:0/3509436159 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e400b9e0 con 0x7f75fc09afb0 2024-03-16T20:45:25.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75fb59e640 1 --2- 172.21.15.122:0/3509436159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75e4004d10 0x7f75e4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75fad9d640 1 --2- 172.21.15.122:0/3509436159 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75fc09afb0 0x7f75e4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75fa59c640 1 --2- 172.21.15.122:0/3509436159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75e4003f00 0x7f75e40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75fb59e640 1 -- 172.21.15.122:0/3509436159 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75fc09afb0 msgr2=0x7f75e4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:25.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75fb59e640 1 --2- 172.21.15.122:0/3509436159 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75fc09afb0 0x7f75e4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75fb59e640 1 -- 172.21.15.122:0/3509436159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75e4003f00 msgr2=0x7f75e40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:25.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75fb59e640 1 --2- 172.21.15.122:0/3509436159 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75e4003f00 0x7f75e40047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75fb59e640 1 -- 172.21.15.122:0/3509436159 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e4003530 con 0x7f75e4004d10 2024-03-16T20:45:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75fb59e640 1 --2- 172.21.15.122:0/3509436159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75e4004d10 0x7f75e4005190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f75f4009fc0 tx=0x7f75f401cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf49942d921b82c4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 -- 172.21.15.122:0/3509436159 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f40a5c50 con 0x7f75e4004d10 2024-03-16T20:45:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 -- 172.21.15.122:0/3509436159 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f400f040 con 0x7f75e4004d10 2024-03-16T20:45:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.020+0000 7f75f9d9b640 1 -- 172.21.15.122:0/3509436159 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f40ad440 con 0x7f75e4004d10 2024-03-16T20:45:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f76010d3640 1 -- 172.21.15.122:0/3509436159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75e4004d10 msgr2=0x7f75e4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f76010d3640 1 --2- 172.21.15.122:0/3509436159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75e4004d10 0x7f75e4005190 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f75f4009fc0 tx=0x7f75f401cc40 comp rx=0 tx=0).stop 2024-03-16T20:45:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f75fb59e640 1 -- 172.21.15.122:0/3509436159 reap_dead start 2024-03-16T20:45:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f76010d3640 1 -- 172.21.15.122:0/3509436159 shutdown_connections 2024-03-16T20:45:25.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f76010d3640 1 -- 172.21.15.122:0/3509436159 >> 172.21.15.122:0/3509436159 conn(0x7f75fc0b8d10 msgr2=0x7f75fc09a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:25.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f76010d3640 1 -- 172.21.15.122:0/3509436159 shutdown_connections 2024-03-16T20:45:25.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f76010d3640 1 -- 172.21.15.122:0/3509436159 wait complete. 2024-03-16T20:45:25.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f76010d3640 1 Processor -- start 2024-03-16T20:45:25.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f76010d3640 1 -- start start 2024-03-16T20:45:25.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f76010d3640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75fc0adb80 0x7f75fc1df940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f76010d3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75fc1dfe80 0x7f75fc1cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f76010d3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75fc1d15c0 0x7f75fc1cb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f76010d3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc0718d0 con 0x7f75fc1d15c0 2024-03-16T20:45:25.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f76010d3640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc071bb0 con 0x7f75fc1dfe80 2024-03-16T20:45:25.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f76010d3640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75fc0aee90 con 0x7f75fc0adb80 2024-03-16T20:45:25.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f75fb59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75fc1d15c0 0x7f75fc1cb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f75fb59e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75fc1d15c0 0x7f75fc1cb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51152/0 (socket says 172.21.15.122:51152) 2024-03-16T20:45:25.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f75fb59e640 1 -- 172.21.15.122:0/3434368134 learned_addr learned my addr 172.21.15.122:0/3434368134 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:25.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f75fad9d640 1 --2- 172.21.15.122:0/3434368134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75fc0adb80 0x7f75fc1df940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f75fb59e640 1 -- 172.21.15.122:0/3434368134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75fc0adb80 msgr2=0x7f75fc1df940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f75fb59e640 1 --2- 172.21.15.122:0/3434368134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75fc0adb80 0x7f75fc1df940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f75fb59e640 1 -- 172.21.15.122:0/3434368134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75fc1dfe80 msgr2=0x7f75fc1cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f75fa59c640 1 --2- 172.21.15.122:0/3434368134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75fc1dfe80 0x7f75fc1cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f75fb59e640 1 --2- 172.21.15.122:0/3434368134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75fc1dfe80 0x7f75fc1cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f75fb59e640 1 -- 172.21.15.122:0/3434368134 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75fc072040 con 0x7f75fc1d15c0 2024-03-16T20:45:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f75fa59c640 1 --2- 172.21.15.122:0/3434368134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75fc1dfe80 0x7f75fc1cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f75fb59e640 1 --2- 172.21.15.122:0/3434368134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75fc1d15c0 0x7f75fc1cb4e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f75f40b0040 tx=0x7f75f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f75e37fe640 1 -- 172.21.15.122:0/3434368134 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f40039a0 con 0x7f75fc1d15c0 2024-03-16T20:45:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f75e37fe640 1 -- 172.21.15.122:0/3434368134 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f4003b00 con 0x7f75fc1d15c0 2024-03-16T20:45:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f76010d3640 1 -- 172.21.15.122:0/3434368134 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75fc1d1b70 con 0x7f75fc1d15c0 2024-03-16T20:45:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f75e37fe640 1 -- 172.21.15.122:0/3434368134 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f4003e70 con 0x7f75fc1d15c0 2024-03-16T20:45:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f76010d3640 1 -- 172.21.15.122:0/3434368134 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75fc0adf60 con 0x7f75fc1d15c0 2024-03-16T20:45:25.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.024+0000 7f76010d3640 1 -- 172.21.15.122:0/3434368134 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75fc1d1d00 con 0x7f75fc1d15c0 2024-03-16T20:45:25.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.028+0000 7f75e37fe640 1 -- 172.21.15.122:0/3434368134 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f75f40b4040 con 0x7f75fc1d15c0 2024-03-16T20:45:25.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.028+0000 7f75e37fe640 1 --2- 172.21.15.122:0/3434368134 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f75e8066e40 0x7f75e8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.028+0000 7f75e37fe640 1 -- 172.21.15.122:0/3434368134 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75f411e580 con 0x7f75fc1d15c0 2024-03-16T20:45:25.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.028+0000 7f75fad9d640 1 --2- 172.21.15.122:0/3434368134 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f75e8066e40 0x7f75e8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.028+0000 7f75fad9d640 1 --2- 172.21.15.122:0/3434368134 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f75e8066e40 0x7f75e8069300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009ab0 tx=0x7f75f0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:25.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.032+0000 7f75e37fe640 1 -- 172.21.15.122:0/3434368134 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f75f40eb4b0 con 0x7f75fc1d15c0 2024-03-16T20:45:25.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.184+0000 7f76010d3640 1 -- 172.21.15.122:0/3434368134 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f75fc098960 con 0x7f75fc1d15c0 2024-03-16T20:45:25.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.184+0000 7f75e37fe640 1 -- 172.21.15.122:0/3434368134 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f75f40eeb70 con 0x7f75fc1d15c0 2024-03-16T20:45:25.188 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-03-16T20:45:25.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.184+0000 7f76010d3640 1 -- 172.21.15.122:0/3434368134 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f75e8066e40 msgr2=0x7f75e8069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.184+0000 7f76010d3640 1 --2- 172.21.15.122:0/3434368134 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f75e8066e40 0x7f75e8069300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009ab0 tx=0x7f75f0008040 comp rx=0 tx=0).stop 2024-03-16T20:45:25.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.184+0000 7f76010d3640 1 -- 172.21.15.122:0/3434368134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75fc1d15c0 msgr2=0x7f75fc1cb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.184+0000 7f76010d3640 1 --2- 172.21.15.122:0/3434368134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75fc1d15c0 0x7f75fc1cb4e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f75f40b0040 tx=0x7f75f4009750 comp rx=0 tx=0).stop 2024-03-16T20:45:25.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.184+0000 7f76010d3640 1 -- 172.21.15.122:0/3434368134 shutdown_connections 2024-03-16T20:45:25.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.184+0000 7f76010d3640 1 --2- 172.21.15.122:0/3434368134 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f75e8066e40 0x7f75e8069300 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.184+0000 7f76010d3640 1 --2- 172.21.15.122:0/3434368134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f75fc1d15c0 0x7f75fc1cb4e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.184+0000 7f76010d3640 1 --2- 172.21.15.122:0/3434368134 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f75fc1dfe80 0x7f75fc1cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.188+0000 7f76010d3640 1 --2- 172.21.15.122:0/3434368134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f75fc0adb80 0x7f75fc1df940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.190 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.188+0000 7f76010d3640 1 -- 172.21.15.122:0/3434368134 >> 172.21.15.122:0/3434368134 conn(0x7f75fc0b8d10 msgr2=0x7f75fc09b570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:25.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.188+0000 7f76010d3640 1 -- 172.21.15.122:0/3434368134 shutdown_connections 2024-03-16T20:45:25.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.188+0000 7f76010d3640 1 -- 172.21.15.122:0/3434368134 wait complete. 2024-03-16T20:45:25.258 INFO:tasks.cephfs_test_runner:Starting test: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) Test that when cd is issued with an argument, CWD is changed 2024-03-16T20:45:25.258 INFO:tasks.cephfs_test_runner:test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-16T20:45:25.258 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:25.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7258009640 1 Processor -- start 2024-03-16T20:45:25.383 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7258009640 1 -- start start 2024-03-16T20:45:25.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7258009640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f725006b1f0 0x7f7250147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7258009640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f725006f760 con 0x7f7250141660 2024-03-16T20:45:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7258009640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7250147730 con 0x7f72501411e0 2024-03-16T20:45:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7258009640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7250141ab0 con 0x7f725006b1f0 2024-03-16T20:45:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f72565a6640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f7250141660 0x7f725014afb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:51808/0 (socket says 172.21.15.122:51808) 2024-03-16T20:45:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f72565a6640 1 -- 172.21.15.122:0/1515240072 learned_addr learned my addr 172.21.15.122:0/1515240072 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7255da5640 1 --2- 172.21.15.122:0/1515240072 >> v2:172.21.15.195:3300/0 conn(0x7f725006b1f0 0x7f7250147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7254da3640 1 -- 172.21.15.122:0/1515240072 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193123550 0 0) 0x7f725006f760 con 0x7f7250141660 2024-03-16T20:45:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7254da3640 1 -- 172.21.15.122:0/1515240072 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f723c003530 con 0x7f7250141660 2024-03-16T20:45:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7254da3640 1 -- 172.21.15.122:0/1515240072 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4092385108 0 0) 0x7f723c003530 con 0x7f7250141660 2024-03-16T20:45:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7254da3640 1 -- 172.21.15.122:0/1515240072 >> v2:172.21.15.195:3300/0 conn(0x7f725006b1f0 msgr2=0x7f7250147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:25.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7254da3640 1 --2- 172.21.15.122:0/1515240072 >> v2:172.21.15.195:3300/0 conn(0x7f725006b1f0 0x7f7250147bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7254da3640 1 -- 172.21.15.122:0/1515240072 >> v1:172.21.15.195:6789/0 conn(0x7f72501411e0 legacy=0x7f725014a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7254da3640 1 -- 172.21.15.122:0/1515240072 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725006f2d0 con 0x7f7250141660 2024-03-16T20:45:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7254da3640 1 -- 172.21.15.122:0/1515240072 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f7248001a00 con 0x7f7250141660 2024-03-16T20:45:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7254da3640 1 -- 172.21.15.122:0/1515240072 >> v1:172.21.15.122:6789/0 conn(0x7f7250141660 legacy=0x7f725014afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7254da3640 1 --2- 172.21.15.122:0/1515240072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f723c003980 0x7f723c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7254da3640 1 --2- 172.21.15.122:0/1515240072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f723c0047d0 0x7f723c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7254da3640 1 --2- 172.21.15.122:0/1515240072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f723c005190 0x7f723c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7254da3640 1 -- 172.21.15.122:0/1515240072 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723c00c000 con 0x7f723c003980 2024-03-16T20:45:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7254da3640 1 -- 172.21.15.122:0/1515240072 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f723c00c170 con 0x7f723c0047d0 2024-03-16T20:45:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7254da3640 1 -- 172.21.15.122:0/1515240072 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f723c00c330 con 0x7f723c005190 2024-03-16T20:45:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7255da5640 1 --2- 172.21.15.122:0/1515240072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f723c003980 0x7f723c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f72555a4640 1 --2- 172.21.15.122:0/1515240072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f723c0047d0 0x7f723c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f72565a6640 1 --2- 172.21.15.122:0/1515240072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f723c005190 0x7f723c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7255da5640 1 -- 172.21.15.122:0/1515240072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f723c005190 msgr2=0x7f723c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:25.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7255da5640 1 --2- 172.21.15.122:0/1515240072 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f723c005190 0x7f723c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7255da5640 1 -- 172.21.15.122:0/1515240072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f723c0047d0 msgr2=0x7f723c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:25.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7255da5640 1 --2- 172.21.15.122:0/1515240072 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f723c0047d0 0x7f723c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f72565a6640 1 -- 172.21.15.122:0/1515240072 reap_dead start 2024-03-16T20:45:25.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.380+0000 7f7255da5640 1 -- 172.21.15.122:0/1515240072 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f723c003530 con 0x7f723c003980 2024-03-16T20:45:25.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7255da5640 1 --2- 172.21.15.122:0/1515240072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f723c003980 0x7f723c003da0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f724c009ae0 tx=0x7f724c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ecd5dafbe7bca server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:25.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7254da3640 1 -- 172.21.15.122:0/1515240072 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c0a6450 con 0x7f723c003980 2024-03-16T20:45:25.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7254da3640 1 -- 172.21.15.122:0/1515240072 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724c0a65b0 con 0x7f723c003980 2024-03-16T20:45:25.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7254da3640 1 -- 172.21.15.122:0/1515240072 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724c0a6960 con 0x7f723c003980 2024-03-16T20:45:25.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7258009640 1 -- 172.21.15.122:0/1515240072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f723c003980 msgr2=0x7f723c003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7258009640 1 --2- 172.21.15.122:0/1515240072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f723c003980 0x7f723c003da0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f724c009ae0 tx=0x7f724c01ca70 comp rx=0 tx=0).stop 2024-03-16T20:45:25.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7258009640 1 -- 172.21.15.122:0/1515240072 shutdown_connections 2024-03-16T20:45:25.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7258009640 1 --2- 172.21.15.122:0/1515240072 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f723c003980 0x7f723c003da0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7258009640 1 -- 172.21.15.122:0/1515240072 >> 172.21.15.122:0/1515240072 conn(0x7f7250070d30 msgr2=0x7f7250071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:25.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7258009640 1 -- 172.21.15.122:0/1515240072 shutdown_connections 2024-03-16T20:45:25.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7258009640 1 -- 172.21.15.122:0/1515240072 wait complete. 2024-03-16T20:45:25.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7258009640 1 Processor -- start 2024-03-16T20:45:25.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7258009640 1 -- start start 2024-03-16T20:45:25.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7258009640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f723c003980 0x7f725013e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7258009640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f725013f690 0x7f725013faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7258009640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72501dfa70 0x7f72501dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7258009640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72501e0430 con 0x7f72501dfa70 2024-03-16T20:45:25.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7258009640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72501df4b0 con 0x7f723c003980 2024-03-16T20:45:25.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f7258009640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72500b4d30 con 0x7f725013f690 2024-03-16T20:45:25.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f72565a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72501dfa70 0x7f72501dfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f72565a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72501dfa70 0x7f72501dfef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51156/0 (socket says 172.21.15.122:51156) 2024-03-16T20:45:25.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.384+0000 7f72565a6640 1 -- 172.21.15.122:0/2685067799 learned_addr learned my addr 172.21.15.122:0/2685067799 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:25.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f7255da5640 1 --2- 172.21.15.122:0/2685067799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f723c003980 0x7f725013e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f72555a4640 1 --2- 172.21.15.122:0/2685067799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f725013f690 0x7f725013faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f72565a6640 1 -- 172.21.15.122:0/2685067799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f725013f690 msgr2=0x7f725013faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f72565a6640 1 --2- 172.21.15.122:0/2685067799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f725013f690 0x7f725013faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f72565a6640 1 -- 172.21.15.122:0/2685067799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f723c003980 msgr2=0x7f725013e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f72565a6640 1 --2- 172.21.15.122:0/2685067799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f723c003980 0x7f725013e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f72565a6640 1 -- 172.21.15.122:0/2685067799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72500b76d0 con 0x7f72501dfa70 2024-03-16T20:45:25.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f7255da5640 1 --2- 172.21.15.122:0/2685067799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f723c003980 0x7f725013e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:25.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f72565a6640 1 --2- 172.21.15.122:0/2685067799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72501dfa70 0x7f72501dfef0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7250141a40 tx=0x7f7248007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:25.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f72467fc640 1 -- 172.21.15.122:0/2685067799 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f724800eb00 con 0x7f72501dfa70 2024-03-16T20:45:25.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f72467fc640 1 -- 172.21.15.122:0/2685067799 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f724800ec60 con 0x7f72501dfa70 2024-03-16T20:45:25.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f7258009640 1 -- 172.21.15.122:0/2685067799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72500b4a50 con 0x7f72501dfa70 2024-03-16T20:45:25.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f72467fc640 1 -- 172.21.15.122:0/2685067799 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7248013420 con 0x7f72501dfa70 2024-03-16T20:45:25.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f7258009640 1 -- 172.21.15.122:0/2685067799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72501d46c0 con 0x7f72501dfa70 2024-03-16T20:45:25.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f7258009640 1 -- 172.21.15.122:0/2685067799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72500b7530 con 0x7f72501dfa70 2024-03-16T20:45:25.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f72467fc640 1 -- 172.21.15.122:0/2685067799 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f7248013580 con 0x7f72501dfa70 2024-03-16T20:45:25.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f72467fc640 1 --2- 172.21.15.122:0/2685067799 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7234066e40 0x7f7234069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f72467fc640 1 -- 172.21.15.122:0/2685067799 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7248084da0 con 0x7f72501dfa70 2024-03-16T20:45:25.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f7255da5640 1 --2- 172.21.15.122:0/2685067799 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7234066e40 0x7f7234069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.388+0000 7f7255da5640 1 --2- 172.21.15.122:0/2685067799 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7234066e40 0x7f7234069300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f724c0ad000 tx=0x7f724c0b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:25.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.392+0000 7f72467fc640 1 -- 172.21.15.122:0/2685067799 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f72500b7530 con 0x7f72501dfa70 2024-03-16T20:45:25.556 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.552+0000 7f7258009640 1 -- 172.21.15.122:0/2685067799 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72500b7530 con 0x7f72501dfa70 2024-03-16T20:45:25.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.556+0000 7f72467fc640 1 -- 172.21.15.122:0/2685067799 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f72500b7530 con 0x7f72501dfa70 2024-03-16T20:45:25.559 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:25.560 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:06.404479+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:25.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.556+0000 7f7258009640 1 -- 172.21.15.122:0/2685067799 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7234066e40 msgr2=0x7f7234069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.556+0000 7f7258009640 1 --2- 172.21.15.122:0/2685067799 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7234066e40 0x7f7234069300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f724c0ad000 tx=0x7f724c0b2040 comp rx=0 tx=0).stop 2024-03-16T20:45:25.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.556+0000 7f7258009640 1 -- 172.21.15.122:0/2685067799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72501dfa70 msgr2=0x7f72501dfef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.556+0000 7f7258009640 1 --2- 172.21.15.122:0/2685067799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72501dfa70 0x7f72501dfef0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7250141a40 tx=0x7f7248007bc0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.556+0000 7f7258009640 1 -- 172.21.15.122:0/2685067799 shutdown_connections 2024-03-16T20:45:25.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.556+0000 7f7258009640 1 --2- 172.21.15.122:0/2685067799 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7234066e40 0x7f7234069300 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.556+0000 7f7258009640 1 --2- 172.21.15.122:0/2685067799 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72501dfa70 0x7f72501dfef0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.556+0000 7f7258009640 1 --2- 172.21.15.122:0/2685067799 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f725013f690 0x7f725013faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.556+0000 7f7258009640 1 --2- 172.21.15.122:0/2685067799 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f723c003980 0x7f725013e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.556+0000 7f7258009640 1 -- 172.21.15.122:0/2685067799 >> 172.21.15.122:0/2685067799 conn(0x7f7250070d30 msgr2=0x7f7250139b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:25.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.556+0000 7f7258009640 1 -- 172.21.15.122:0/2685067799 shutdown_connections 2024-03-16T20:45:25.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.556+0000 7f7258009640 1 -- 172.21.15.122:0/2685067799 wait complete. 2024-03-16T20:45:25.579 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T20:44:52.653523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T20:45:02.758515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T20:45:03.710783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T20:45:25.579 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T20:45:25.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.696+0000 7f1cf5488640 1 Processor -- start 2024-03-16T20:45:25.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cf5488640 1 -- start start 2024-03-16T20:45:25.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cf5488640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1cf006b1f0 0x7f1cf00b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cf5488640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1cf006f6f0 0x7f1cf0141270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cf5488640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cf0147ca0 con 0x7f1cf01417b0 2024-03-16T20:45:25.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cf5488640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf006b5d0 con 0x7f1cf006f6f0 2024-03-16T20:45:25.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cf5488640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf0147650 con 0x7f1cf006b1f0 2024-03-16T20:45:25.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cee7fc640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1cf006f6f0 0x7f1cf0141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cef7fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f1cf01417b0 0x7f1cf014a870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:51812/0 (socket says 172.21.15.122:51812) 2024-03-16T20:45:25.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cef7fe640 1 -- 172.21.15.122:0/2811664833 learned_addr learned my addr 172.21.15.122:0/2811664833 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:25.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1ceeffd640 1 --2- 172.21.15.122:0/2811664833 >> v2:172.21.15.195:3300/0 conn(0x7f1cf006b1f0 0x7f1cf00b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cee7fc640 1 -- 172.21.15.122:0/2811664833 >> v2:172.21.15.195:3300/0 conn(0x7f1cf006b1f0 msgr2=0x7f1cf00b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:25.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cee7fc640 1 --2- 172.21.15.122:0/2811664833 >> v2:172.21.15.195:3300/0 conn(0x7f1cf006b1f0 0x7f1cf00b7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cee7fc640 1 -- 172.21.15.122:0/2811664833 >> v1:172.21.15.122:6789/0 conn(0x7f1cf01417b0 legacy=0x7f1cf014a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cedffb640 1 -- 172.21.15.122:0/2811664833 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801155645 0 0) 0x7f1cf0147ca0 con 0x7f1cf01417b0 2024-03-16T20:45:25.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cee7fc640 1 -- 172.21.15.122:0/2811664833 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf0147930 con 0x7f1cf006f6f0 2024-03-16T20:45:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cee7fc640 1 --2- 172.21.15.122:0/2811664833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1cf006f6f0 0x7f1cf0141270 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0067cd0 tx=0x7f1ce401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b504fddb615b7ad2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cedffb640 1 -- 172.21.15.122:0/2811664833 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40ab600 con 0x7f1cf006f6f0 2024-03-16T20:45:25.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cedffb640 1 -- 172.21.15.122:0/2811664833 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce40a7050 con 0x7f1cf006f6f0 2024-03-16T20:45:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.700+0000 7f1cedffb640 1 -- 172.21.15.122:0/2811664833 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40ab980 con 0x7f1cf006f6f0 2024-03-16T20:45:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 -- 172.21.15.122:0/2811664833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1cf006f6f0 msgr2=0x7f1cf0141270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 --2- 172.21.15.122:0/2811664833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1cf006f6f0 0x7f1cf0141270 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1cf0067cd0 tx=0x7f1ce401c420 comp rx=0 tx=0).stop 2024-03-16T20:45:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 -- 172.21.15.122:0/2811664833 shutdown_connections 2024-03-16T20:45:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 --2- 172.21.15.122:0/2811664833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1cf006f6f0 0x7f1cf0141270 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 --2- 172.21.15.122:0/2811664833 >> v2:172.21.15.195:3300/0 conn(0x7f1cf006b1f0 0x7f1cf00b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 -- 172.21.15.122:0/2811664833 >> 172.21.15.122:0/2811664833 conn(0x7f1cf0070d30 msgr2=0x7f1cf0071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 -- 172.21.15.122:0/2811664833 shutdown_connections 2024-03-16T20:45:25.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 -- 172.21.15.122:0/2811664833 wait complete. 2024-03-16T20:45:25.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 Processor -- start 2024-03-16T20:45:25.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 -- start start 2024-03-16T20:45:25.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1cf006b1f0 0x7f1cf013d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cf006f6f0 0x7f1cf0143cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cf01417b0 0x7f1cf0144200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf00ad210 con 0x7f1cf006b1f0 2024-03-16T20:45:25.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf00ad4f0 con 0x7f1cf01417b0 2024-03-16T20:45:25.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf0142010 con 0x7f1cf006f6f0 2024-03-16T20:45:25.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1ceeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1cf006b1f0 0x7f1cf013d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1ceeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1cf006b1f0 0x7f1cf013d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51164/0 (socket says 172.21.15.122:51164) 2024-03-16T20:45:25.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1ceeffd640 1 -- 172.21.15.122:0/998439849 learned_addr learned my addr 172.21.15.122:0/998439849 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:25.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cef7fe640 1 --2- 172.21.15.122:0/998439849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cf01417b0 0x7f1cf0144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cee7fc640 1 --2- 172.21.15.122:0/998439849 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cf006f6f0 0x7f1cf0143cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1ceeffd640 1 -- 172.21.15.122:0/998439849 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cf006f6f0 msgr2=0x7f1cf0143cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1ceeffd640 1 --2- 172.21.15.122:0/998439849 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cf006f6f0 0x7f1cf0143cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1ceeffd640 1 -- 172.21.15.122:0/998439849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cf01417b0 msgr2=0x7f1cf0144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1ceeffd640 1 --2- 172.21.15.122:0/998439849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cf01417b0 0x7f1cf0144200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1ceeffd640 1 -- 172.21.15.122:0/998439849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce4009420 con 0x7f1cf006b1f0 2024-03-16T20:45:25.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1ceeffd640 1 --2- 172.21.15.122:0/998439849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1cf006b1f0 0x7f1cf013d1d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000d5f0 tx=0x7f1ce000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:25.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cd77fe640 1 -- 172.21.15.122:0/998439849 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce00125f0 con 0x7f1cf006b1f0 2024-03-16T20:45:25.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cd77fe640 1 -- 172.21.15.122:0/998439849 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ce0012750 con 0x7f1cf006b1f0 2024-03-16T20:45:25.711 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 -- 172.21.15.122:0/998439849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf01431f0 con 0x7f1cf006b1f0 2024-03-16T20:45:25.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cd77fe640 1 -- 172.21.15.122:0/998439849 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0012ae0 con 0x7f1cf006b1f0 2024-03-16T20:45:25.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 -- 172.21.15.122:0/998439849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf0144b00 con 0x7f1cf006b1f0 2024-03-16T20:45:25.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.704+0000 7f1cf5488640 1 -- 172.21.15.122:0/998439849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf00ba6b0 con 0x7f1cf006b1f0 2024-03-16T20:45:25.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.708+0000 7f1cd77fe640 1 -- 172.21.15.122:0/998439849 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f1ce0012c40 con 0x7f1cf006b1f0 2024-03-16T20:45:25.712 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.708+0000 7f1cd77fe640 1 --2- 172.21.15.122:0/998439849 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1cdc066e40 0x7f1cdc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:25.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.708+0000 7f1cd77fe640 1 -- 172.21.15.122:0/998439849 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ce0084ea0 con 0x7f1cf006b1f0 2024-03-16T20:45:25.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.708+0000 7f1cee7fc640 1 --2- 172.21.15.122:0/998439849 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1cdc066e40 0x7f1cdc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:25.713 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.708+0000 7f1cee7fc640 1 --2- 172.21.15.122:0/998439849 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1cdc066e40 0x7f1cdc069300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40ad000 tx=0x7f1ce4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:25.714 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.712+0000 7f1cd77fe640 1 -- 172.21.15.122:0/998439849 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1ce0051d20 con 0x7f1cf006b1f0 2024-03-16T20:45:25.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.868+0000 7f1cf5488640 1 -- 172.21.15.122:0/998439849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1cf0143900 con 0x7f1cf006b1f0 2024-03-16T20:45:25.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.872+0000 7f1cd77fe640 1 -- 172.21.15.122:0/998439849 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ce00553e0 con 0x7f1cf006b1f0 2024-03-16T20:45:25.874 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-03-16T20:45:25.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.872+0000 7f1cf5488640 1 -- 172.21.15.122:0/998439849 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1cdc066e40 msgr2=0x7f1cdc069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.872+0000 7f1cf5488640 1 --2- 172.21.15.122:0/998439849 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1cdc066e40 0x7f1cdc069300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40ad000 tx=0x7f1ce4008040 comp rx=0 tx=0).stop 2024-03-16T20:45:25.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.872+0000 7f1cf5488640 1 -- 172.21.15.122:0/998439849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1cf006b1f0 msgr2=0x7f1cf013d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:25.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.872+0000 7f1cf5488640 1 --2- 172.21.15.122:0/998439849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1cf006b1f0 0x7f1cf013d1d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000d5f0 tx=0x7f1ce000dab0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.872+0000 7f1cf5488640 1 -- 172.21.15.122:0/998439849 shutdown_connections 2024-03-16T20:45:25.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.872+0000 7f1cf5488640 1 --2- 172.21.15.122:0/998439849 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1cdc066e40 0x7f1cdc069300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.872+0000 7f1cf5488640 1 --2- 172.21.15.122:0/998439849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1cf01417b0 0x7f1cf0144200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.872+0000 7f1cf5488640 1 --2- 172.21.15.122:0/998439849 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1cf006f6f0 0x7f1cf0143cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.872+0000 7f1cf5488640 1 --2- 172.21.15.122:0/998439849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1cf006b1f0 0x7f1cf013d1d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:25.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.872+0000 7f1cf5488640 1 -- 172.21.15.122:0/998439849 >> 172.21.15.122:0/998439849 conn(0x7f1cf0070d30 msgr2=0x7f1cf0071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:25.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.872+0000 7f1cf5488640 1 -- 172.21.15.122:0/998439849 shutdown_connections 2024-03-16T20:45:25.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:25.872+0000 7f1cf5488640 1 -- 172.21.15.122:0/998439849 wait complete. 2024-03-16T20:45:25.893 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T20:45:26.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.020+0000 7f5cfc16d640 1 Processor -- start 2024-03-16T20:45:26.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.020+0000 7f5cfc16d640 1 -- start start 2024-03-16T20:45:26.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.020+0000 7f5cfc16d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5cf40bb1d0 0x7f5cf406cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.020+0000 7f5cfc16d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf4147ff0 con 0x7f5cf406d180 2024-03-16T20:45:26.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.020+0000 7f5cfc16d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cf41479a0 con 0x7f5cf4141580 2024-03-16T20:45:26.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.020+0000 7f5cfc16d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cf4147c80 con 0x7f5cf40bb1d0 2024-03-16T20:45:26.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.020+0000 7f5cf9708640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5cf40bb1d0 0x7f5cf406cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.020+0000 7f5cf9708640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f5cf40bb1d0 0x7f5cf406cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51166/0 (socket says 172.21.15.122:51166) 2024-03-16T20:45:26.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.020+0000 7f5cf9708640 1 -- 172.21.15.122:0/4146419504 learned_addr learned my addr 172.21.15.122:0/4146419504 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:26.025 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.020+0000 7f5cfa70a640 1 --1- 172.21.15.122:0/4146419504 >> v1:172.21.15.122:6789/0 conn(0x7f5cf406d180 0x7f5cf4151920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:51826/0 (socket says 172.21.15.122:51826) 2024-03-16T20:45:26.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.020+0000 7f5cf9708640 1 -- 172.21.15.122:0/4146419504 >> v1:172.21.15.195:6789/0 conn(0x7f5cf4141580 legacy=0x7f5cf40baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:26.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.020+0000 7f5cf9708640 1 -- 172.21.15.122:0/4146419504 >> v1:172.21.15.122:6789/0 conn(0x7f5cf406d180 legacy=0x7f5cf4151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:26.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.020+0000 7f5cf8f07640 1 -- 172.21.15.122:0/4146419504 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2740197375 0 0) 0x7f5cf4147ff0 con 0x7f5cf406d180 2024-03-16T20:45:26.026 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.020+0000 7f5cf9708640 1 -- 172.21.15.122:0/4146419504 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf41475b0 con 0x7f5cf40bb1d0 2024-03-16T20:45:26.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.024+0000 7f5cf9708640 1 --2- 172.21.15.122:0/4146419504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf40bb1d0 0x7f5cf406cc40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009870 tx=0x7f5ce401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb48d84fe9c34b04 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:26.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.024+0000 7f5cf8f07640 1 -- 172.21.15.122:0/4146419504 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40ad660 con 0x7f5cf40bb1d0 2024-03-16T20:45:26.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.024+0000 7f5cf8f07640 1 -- 172.21.15.122:0/4146419504 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ce40ad7c0 con 0x7f5cf40bb1d0 2024-03-16T20:45:26.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.024+0000 7f5cf8f07640 1 -- 172.21.15.122:0/4146419504 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40adb50 con 0x7f5cf40bb1d0 2024-03-16T20:45:26.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.024+0000 7f5cfc16d640 1 -- 172.21.15.122:0/4146419504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf40bb1d0 msgr2=0x7f5cf406cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:26.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.024+0000 7f5cfc16d640 1 --2- 172.21.15.122:0/4146419504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf40bb1d0 0x7f5cf406cc40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009870 tx=0x7f5ce401c560 comp rx=0 tx=0).stop 2024-03-16T20:45:26.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.024+0000 7f5cfc16d640 1 -- 172.21.15.122:0/4146419504 shutdown_connections 2024-03-16T20:45:26.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.024+0000 7f5cfc16d640 1 --2- 172.21.15.122:0/4146419504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf40bb1d0 0x7f5cf406cc40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.024+0000 7f5cfc16d640 1 -- 172.21.15.122:0/4146419504 >> 172.21.15.122:0/4146419504 conn(0x7f5cf40bca20 msgr2=0x7f5cf40bce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:26.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.024+0000 7f5cfc16d640 1 -- 172.21.15.122:0/4146419504 shutdown_connections 2024-03-16T20:45:26.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.024+0000 7f5cfc16d640 1 -- 172.21.15.122:0/4146419504 wait complete. 2024-03-16T20:45:26.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.024+0000 7f5cfc16d640 1 Processor -- start 2024-03-16T20:45:26.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.024+0000 7f5cfc16d640 1 -- start start 2024-03-16T20:45:26.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cfc16d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf406d180 0x7f5cf40a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cfc16d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf40bb1d0 0x7f5cf4142ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cfc16d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf4141580 0x7f5cf41454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cfc16d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf4142ac0 con 0x7f5cf406d180 2024-03-16T20:45:26.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cfc16d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf413ab80 con 0x7f5cf4141580 2024-03-16T20:45:26.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cfc16d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf4142770 con 0x7f5cf40bb1d0 2024-03-16T20:45:26.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cf9f09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf406d180 0x7f5cf40a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cf9f09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf406d180 0x7f5cf40a1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51170/0 (socket says 172.21.15.122:51170) 2024-03-16T20:45:26.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cf9f09640 1 -- 172.21.15.122:0/447591425 learned_addr learned my addr 172.21.15.122:0/447591425 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:26.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cf9708640 1 --2- 172.21.15.122:0/447591425 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf40bb1d0 0x7f5cf4142ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cfa70a640 1 --2- 172.21.15.122:0/447591425 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf4141580 0x7f5cf41454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cf9f09640 1 -- 172.21.15.122:0/447591425 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf40bb1d0 msgr2=0x7f5cf4142ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:26.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cf9f09640 1 --2- 172.21.15.122:0/447591425 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf40bb1d0 0x7f5cf4142ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cf9f09640 1 -- 172.21.15.122:0/447591425 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf4141580 msgr2=0x7f5cf41454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:26.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cf9f09640 1 --2- 172.21.15.122:0/447591425 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf4141580 0x7f5cf41454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cf9f09640 1 -- 172.21.15.122:0/447591425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce40094d0 con 0x7f5cf406d180 2024-03-16T20:45:26.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cf9f09640 1 --2- 172.21.15.122:0/447591425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf406d180 0x7f5cf40a1240 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000c3c0 tx=0x7f5cf000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:26.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cea7fc640 1 -- 172.21.15.122:0/447591425 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf00052d0 con 0x7f5cf406d180 2024-03-16T20:45:26.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cea7fc640 1 -- 172.21.15.122:0/447591425 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf0005430 con 0x7f5cf406d180 2024-03-16T20:45:26.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cfc16d640 1 -- 172.21.15.122:0/447591425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf4144390 con 0x7f5cf406d180 2024-03-16T20:45:26.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cea7fc640 1 -- 172.21.15.122:0/447591425 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0003840 con 0x7f5cf406d180 2024-03-16T20:45:26.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cfc16d640 1 -- 172.21.15.122:0/447591425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf41423e0 con 0x7f5cf406d180 2024-03-16T20:45:26.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cfc16d640 1 -- 172.21.15.122:0/447591425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf40684d0 con 0x7f5cf406d180 2024-03-16T20:45:26.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cea7fc640 1 -- 172.21.15.122:0/447591425 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f5cf00039a0 con 0x7f5cf406d180 2024-03-16T20:45:26.033 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cea7fc640 1 --2- 172.21.15.122:0/447591425 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5cd8066e70 0x7f5cd8069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cea7fc640 1 -- 172.21.15.122:0/447591425 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5cf0084050 con 0x7f5cf406d180 2024-03-16T20:45:26.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.028+0000 7f5cf9708640 1 --2- 172.21.15.122:0/447591425 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5cd8066e70 0x7f5cd8069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.034 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.032+0000 7f5cf9708640 1 --2- 172.21.15.122:0/447591425 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5cd8066e70 0x7f5cd8069330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40af000 tx=0x7f5ce4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:26.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.032+0000 7f5cea7fc640 1 -- 172.21.15.122:0/447591425 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5cf0050fb0 con 0x7f5cf406d180 2024-03-16T20:45:26.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.192+0000 7f5cfc16d640 1 -- 172.21.15.122:0/447591425 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5cf40bd6e0 con 0x7f5cf406d180 2024-03-16T20:45:26.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.192+0000 7f5cea7fc640 1 -- 172.21.15.122:0/447591425 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5cf40bd6e0 con 0x7f5cf406d180 2024-03-16T20:45:26.195 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-03-16T20:45:26.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.192+0000 7f5cfc16d640 1 -- 172.21.15.122:0/447591425 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5cd8066e70 msgr2=0x7f5cd8069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:26.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.192+0000 7f5cfc16d640 1 --2- 172.21.15.122:0/447591425 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5cd8066e70 0x7f5cd8069330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40af000 tx=0x7f5ce4008040 comp rx=0 tx=0).stop 2024-03-16T20:45:26.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.192+0000 7f5cfc16d640 1 -- 172.21.15.122:0/447591425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf406d180 msgr2=0x7f5cf40a1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:26.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.192+0000 7f5cfc16d640 1 --2- 172.21.15.122:0/447591425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf406d180 0x7f5cf40a1240 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000c3c0 tx=0x7f5cf000c880 comp rx=0 tx=0).stop 2024-03-16T20:45:26.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.192+0000 7f5cfc16d640 1 -- 172.21.15.122:0/447591425 shutdown_connections 2024-03-16T20:45:26.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.192+0000 7f5cfc16d640 1 --2- 172.21.15.122:0/447591425 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5cd8066e70 0x7f5cd8069330 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.192+0000 7f5cfc16d640 1 --2- 172.21.15.122:0/447591425 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5cf4141580 0x7f5cf41454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.192+0000 7f5cfc16d640 1 --2- 172.21.15.122:0/447591425 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5cf40bb1d0 0x7f5cf4142ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.192+0000 7f5cfc16d640 1 --2- 172.21.15.122:0/447591425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5cf406d180 0x7f5cf40a1240 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.192+0000 7f5cfc16d640 1 -- 172.21.15.122:0/447591425 >> 172.21.15.122:0/447591425 conn(0x7f5cf40bca20 msgr2=0x7f5cf413dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:26.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.192+0000 7f5cfc16d640 1 -- 172.21.15.122:0/447591425 shutdown_connections 2024-03-16T20:45:26.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.192+0000 7f5cfc16d640 1 -- 172.21.15.122:0/447591425 wait complete. 2024-03-16T20:45:26.212 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T20:45:26.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9bea3b8640 1 Processor -- start 2024-03-16T20:45:26.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9bea3b8640 1 -- start start 2024-03-16T20:45:26.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9bea3b8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9be4141580 0x7f9be40baac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9bea3b8640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9be40bb000 0x7f9be406cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9bea3b8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9be406d180 0x7f9be414a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9bea3b8640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be4147ff0 con 0x7f9be406d180 2024-03-16T20:45:26.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9bea3b8640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be40bb460 con 0x7f9be40bb000 2024-03-16T20:45:26.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9bea3b8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be41479a0 con 0x7f9be4141580 2024-03-16T20:45:26.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9be8955640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9be406d180 0x7f9be414a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9be8955640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9be406d180 0x7f9be414a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51172/0 (socket says 172.21.15.122:51172) 2024-03-16T20:45:26.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9be8955640 1 -- 172.21.15.122:0/1087635637 learned_addr learned my addr 172.21.15.122:0/1087635637 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:26.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9bdbfff640 1 --2- 172.21.15.122:0/1087635637 >> v2:172.21.15.195:3300/0 conn(0x7f9be40bb000 0x7f9be406cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9be3fff640 1 --2- 172.21.15.122:0/1087635637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9be4141580 0x7f9be40baac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9be8955640 1 -- 172.21.15.122:0/1087635637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9be4141580 msgr2=0x7f9be40baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9be8955640 1 --2- 172.21.15.122:0/1087635637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9be4141580 0x7f9be40baac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9be8955640 1 -- 172.21.15.122:0/1087635637 >> v2:172.21.15.195:3300/0 conn(0x7f9be40bb000 msgr2=0x7f9be406cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9be8955640 1 --2- 172.21.15.122:0/1087635637 >> v2:172.21.15.195:3300/0 conn(0x7f9be40bb000 0x7f9be406cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9be8955640 1 -- 172.21.15.122:0/1087635637 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be4147c80 con 0x7f9be406d180 2024-03-16T20:45:26.325 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.320+0000 7f9be3fff640 1 --2- 172.21.15.122:0/1087635637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9be4141580 0x7f9be40baac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:26.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9be8955640 1 --2- 172.21.15.122:0/1087635637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be406d180 0x7f9be414a5d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9be40a2c30 tx=0x7f9bdc01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51823892b298ac0d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:26.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9be37fe640 1 -- 172.21.15.122:0/1087635637 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc0a5670 con 0x7f9be406d180 2024-03-16T20:45:26.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9be37fe640 1 -- 172.21.15.122:0/1087635637 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bdc0a57d0 con 0x7f9be406d180 2024-03-16T20:45:26.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9be37fe640 1 -- 172.21.15.122:0/1087635637 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc0a5b60 con 0x7f9be406d180 2024-03-16T20:45:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 -- 172.21.15.122:0/1087635637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be406d180 msgr2=0x7f9be414a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 --2- 172.21.15.122:0/1087635637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be406d180 0x7f9be414a5d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9be40a2c30 tx=0x7f9bdc01c9f0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 -- 172.21.15.122:0/1087635637 shutdown_connections 2024-03-16T20:45:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 --2- 172.21.15.122:0/1087635637 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be406d180 0x7f9be414a5d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 --2- 172.21.15.122:0/1087635637 >> v2:172.21.15.195:3300/0 conn(0x7f9be40bb000 0x7f9be406cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 --2- 172.21.15.122:0/1087635637 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9be4141580 0x7f9be40baac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.327 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 -- 172.21.15.122:0/1087635637 >> 172.21.15.122:0/1087635637 conn(0x7f9be40bca20 msgr2=0x7f9be40bce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:26.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 -- 172.21.15.122:0/1087635637 shutdown_connections 2024-03-16T20:45:26.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 -- 172.21.15.122:0/1087635637 wait complete. 2024-03-16T20:45:26.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 Processor -- start 2024-03-16T20:45:26.328 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 -- start start 2024-03-16T20:45:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be406d180 0x7f9be4141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9be40bb000 0x7f9be409fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9be4141580 0x7f9be4144f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be4142290 con 0x7f9be406d180 2024-03-16T20:45:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be41424a0 con 0x7f9be4141580 2024-03-16T20:45:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be4142780 con 0x7f9be40bb000 2024-03-16T20:45:26.330 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9be3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be406d180 0x7f9be4141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9be3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be406d180 0x7f9be4141d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51188/0 (socket says 172.21.15.122:51188) 2024-03-16T20:45:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9be3fff640 1 -- 172.21.15.122:0/794852385 learned_addr learned my addr 172.21.15.122:0/794852385 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9be8955640 1 --2- 172.21.15.122:0/794852385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9be4141580 0x7f9be4144f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bdbfff640 1 --2- 172.21.15.122:0/794852385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9be40bb000 0x7f9be409fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9be3fff640 1 -- 172.21.15.122:0/794852385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9be40bb000 msgr2=0x7f9be409fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9be3fff640 1 --2- 172.21.15.122:0/794852385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9be40bb000 0x7f9be409fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9be3fff640 1 -- 172.21.15.122:0/794852385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9be4141580 msgr2=0x7f9be4144f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9be3fff640 1 --2- 172.21.15.122:0/794852385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9be4141580 0x7f9be4144f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9be3fff640 1 -- 172.21.15.122:0/794852385 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bdc009b30 con 0x7f9be406d180 2024-03-16T20:45:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9be3fff640 1 --2- 172.21.15.122:0/794852385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be406d180 0x7f9be4141d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400e810 tx=0x7f9bd400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9be17fa640 1 -- 172.21.15.122:0/794852385 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd400c8d0 con 0x7f9be406d180 2024-03-16T20:45:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9be17fa640 1 -- 172.21.15.122:0/794852385 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd400ca30 con 0x7f9be406d180 2024-03-16T20:45:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 -- 172.21.15.122:0/794852385 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be40a1520 con 0x7f9be406d180 2024-03-16T20:45:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9be17fa640 1 -- 172.21.15.122:0/794852385 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd400cde0 con 0x7f9be406d180 2024-03-16T20:45:26.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.324+0000 7f9bea3b8640 1 -- 172.21.15.122:0/794852385 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be40a1190 con 0x7f9be406d180 2024-03-16T20:45:26.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.328+0000 7f9bea3b8640 1 -- 172.21.15.122:0/794852385 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be40a0260 con 0x7f9be406d180 2024-03-16T20:45:26.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.328+0000 7f9be17fa640 1 -- 172.21.15.122:0/794852385 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f9bd4026430 con 0x7f9be406d180 2024-03-16T20:45:26.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.328+0000 7f9be17fa640 1 --2- 172.21.15.122:0/794852385 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9bc8066e40 0x7f9bc8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.328+0000 7f9be17fa640 1 -- 172.21.15.122:0/794852385 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bd4084ab0 con 0x7f9be406d180 2024-03-16T20:45:26.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.328+0000 7f9bdbfff640 1 --2- 172.21.15.122:0/794852385 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9bc8066e40 0x7f9bc8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.328+0000 7f9bdbfff640 1 --2- 172.21.15.122:0/794852385 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9bc8066e40 0x7f9bc8069300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9be413af10 tx=0x7f9bcc008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:26.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.332+0000 7f9be17fa640 1 -- 172.21.15.122:0/794852385 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9bd4051910 con 0x7f9be406d180 2024-03-16T20:45:26.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.476+0000 7f9bea3b8640 1 -- 172.21.15.122:0/794852385 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9be40a0260 con 0x7f9be406d180 2024-03-16T20:45:26.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.476+0000 7f9be17fa640 1 -- 172.21.15.122:0/794852385 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9bd4021020 con 0x7f9be406d180 2024-03-16T20:45:26.481 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-03-16T20:45:26.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.480+0000 7f9bea3b8640 1 -- 172.21.15.122:0/794852385 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9bc8066e40 msgr2=0x7f9bc8069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:26.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.480+0000 7f9bea3b8640 1 --2- 172.21.15.122:0/794852385 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9bc8066e40 0x7f9bc8069300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9be413af10 tx=0x7f9bcc008040 comp rx=0 tx=0).stop 2024-03-16T20:45:26.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.480+0000 7f9bea3b8640 1 -- 172.21.15.122:0/794852385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be406d180 msgr2=0x7f9be4141d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:26.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.480+0000 7f9bea3b8640 1 --2- 172.21.15.122:0/794852385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be406d180 0x7f9be4141d50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9bd400e810 tx=0x7f9bd400ecd0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.480+0000 7f9bea3b8640 1 -- 172.21.15.122:0/794852385 shutdown_connections 2024-03-16T20:45:26.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.480+0000 7f9bea3b8640 1 --2- 172.21.15.122:0/794852385 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9bc8066e40 0x7f9bc8069300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.480+0000 7f9bea3b8640 1 --2- 172.21.15.122:0/794852385 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9be4141580 0x7f9be4144f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.480+0000 7f9bea3b8640 1 --2- 172.21.15.122:0/794852385 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9be40bb000 0x7f9be409fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.480+0000 7f9bea3b8640 1 --2- 172.21.15.122:0/794852385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9be406d180 0x7f9be4141d50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.480+0000 7f9bea3b8640 1 -- 172.21.15.122:0/794852385 >> 172.21.15.122:0/794852385 conn(0x7f9be40bca20 msgr2=0x7f9be40b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:26.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.480+0000 7f9bea3b8640 1 -- 172.21.15.122:0/794852385 shutdown_connections 2024-03-16T20:45:26.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.480+0000 7f9bea3b8640 1 -- 172.21.15.122:0/794852385 wait complete. 2024-03-16T20:45:26.500 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args' 2024-03-16T20:45:26.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3ae6bc7640 1 Processor -- start 2024-03-16T20:45:26.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3ae6bc7640 1 -- start start 2024-03-16T20:45:26.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3ae6bc7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ae00b2690 0x7f3ae00b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3ae6bc7640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae00b46e0 con 0x7f3ae0143dd0 2024-03-16T20:45:26.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3ae6bc7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae00b2a70 con 0x7f3ae0071a20 2024-03-16T20:45:26.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3ae6bc7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae013e5e0 con 0x7f3ae00b2690 2024-03-16T20:45:26.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3ae4963640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ae00b2690 0x7f3ae00b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adffff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f3ae0143dd0 0x7f3ae0147be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:51840/0 (socket says 172.21.15.122:51840) 2024-03-16T20:45:26.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adffff640 1 -- 172.21.15.122:0/3885391334 learned_addr learned my addr 172.21.15.122:0/3885391334 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:26.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adf7fe640 1 -- 172.21.15.122:0/3885391334 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401815955 0 0) 0x7f3ae00b46e0 con 0x7f3ae0143dd0 2024-03-16T20:45:26.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adf7fe640 1 -- 172.21.15.122:0/3885391334 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ac8003530 con 0x7f3ae0143dd0 2024-03-16T20:45:26.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adf7fe640 1 -- 172.21.15.122:0/3885391334 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2686198761 0 0) 0x7f3ac8003530 con 0x7f3ae0143dd0 2024-03-16T20:45:26.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adf7fe640 1 -- 172.21.15.122:0/3885391334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ae00b2690 msgr2=0x7f3ae00b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:26.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adf7fe640 1 --2- 172.21.15.122:0/3885391334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ae00b2690 0x7f3ae00b41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adf7fe640 1 -- 172.21.15.122:0/3885391334 >> v1:172.21.15.195:6789/0 conn(0x7f3ae0071a20 legacy=0x7f3ae01445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:26.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adf7fe640 1 -- 172.21.15.122:0/3885391334 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae00b4950 con 0x7f3ae0143dd0 2024-03-16T20:45:26.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adf7fe640 1 -- 172.21.15.122:0/3885391334 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f3ad0001680 con 0x7f3ae0143dd0 2024-03-16T20:45:26.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adf7fe640 1 -- 172.21.15.122:0/3885391334 >> v1:172.21.15.122:6789/0 conn(0x7f3ae0143dd0 legacy=0x7f3ae0147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:26.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adf7fe640 1 --2- 172.21.15.122:0/3885391334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ac80039b0 0x7f3ac8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adf7fe640 1 --2- 172.21.15.122:0/3885391334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ae00b2690 0x7f3ac80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adf7fe640 1 --2- 172.21.15.122:0/3885391334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ac8004d10 0x7f3ac8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adf7fe640 1 -- 172.21.15.122:0/3885391334 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae013e5e0 con 0x7f3ac8004d10 2024-03-16T20:45:26.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adf7fe640 1 -- 172.21.15.122:0/3885391334 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac800b990 con 0x7f3ac80039b0 2024-03-16T20:45:26.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adf7fe640 1 -- 172.21.15.122:0/3885391334 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac800bb50 con 0x7f3ae00b2690 2024-03-16T20:45:26.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3ae5164640 1 --2- 172.21.15.122:0/3885391334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ac8004d10 0x7f3ac8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3ae4963640 1 --2- 172.21.15.122:0/3885391334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ac80039b0 0x7f3ac8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3adffff640 1 --2- 172.21.15.122:0/3885391334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ae00b2690 0x7f3ac80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3ae5164640 1 -- 172.21.15.122:0/3885391334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ae00b2690 msgr2=0x7f3ac80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:26.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3ae5164640 1 --2- 172.21.15.122:0/3885391334 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ae00b2690 0x7f3ac80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3ae5164640 1 -- 172.21.15.122:0/3885391334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ac80039b0 msgr2=0x7f3ac8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:26.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3ae5164640 1 --2- 172.21.15.122:0/3885391334 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ac80039b0 0x7f3ac8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.600+0000 7f3ae5164640 1 -- 172.21.15.122:0/3885391334 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac8003530 con 0x7f3ac8004d10 2024-03-16T20:45:26.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3ae5164640 1 --2- 172.21.15.122:0/3885391334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ac8004d10 0x7f3ac8005190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8001f60 tx=0x7f3ad801c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6350404cc648089 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:26.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3adf7fe640 1 -- 172.21.15.122:0/3885391334 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad80a6670 con 0x7f3ac8004d10 2024-03-16T20:45:26.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3adf7fe640 1 -- 172.21.15.122:0/3885391334 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad80a67d0 con 0x7f3ac8004d10 2024-03-16T20:45:26.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3adf7fe640 1 -- 172.21.15.122:0/3885391334 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad80a6b80 con 0x7f3ac8004d10 2024-03-16T20:45:26.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3ae6bc7640 1 -- 172.21.15.122:0/3885391334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ac8004d10 msgr2=0x7f3ac8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:26.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3ae6bc7640 1 --2- 172.21.15.122:0/3885391334 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ac8004d10 0x7f3ac8005190 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3ad8001f60 tx=0x7f3ad801c880 comp rx=0 tx=0).stop 2024-03-16T20:45:26.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3ae5164640 1 -- 172.21.15.122:0/3885391334 reap_dead start 2024-03-16T20:45:26.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3ae6bc7640 1 -- 172.21.15.122:0/3885391334 shutdown_connections 2024-03-16T20:45:26.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3ae6bc7640 1 -- 172.21.15.122:0/3885391334 >> 172.21.15.122:0/3885391334 conn(0x7f3ae00b8860 msgr2=0x7f3ae00b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:26.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3ae6bc7640 1 -- 172.21.15.122:0/3885391334 shutdown_connections 2024-03-16T20:45:26.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3ae6bc7640 1 -- 172.21.15.122:0/3885391334 wait complete. 2024-03-16T20:45:26.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3ae6bc7640 1 Processor -- start 2024-03-16T20:45:26.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3ae6bc7640 1 -- start start 2024-03-16T20:45:26.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3ae6bc7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ac8004d10 0x7f3ae00b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3ae6bc7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ae01ce9b0 0x7f3ae01ddb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3ae6bc7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ae013b870 0x7f3ae013bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3ae6bc7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae01cef30 con 0x7f3ae01ce9b0 2024-03-16T20:45:26.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3ae6bc7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae00a8f00 con 0x7f3ae013b870 2024-03-16T20:45:26.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.604+0000 7f3ae6bc7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae01cf1d0 con 0x7f3ac8004d10 2024-03-16T20:45:26.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3adffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ae01ce9b0 0x7f3ae01ddb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3adffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ae01ce9b0 0x7f3ae01ddb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51214/0 (socket says 172.21.15.122:51214) 2024-03-16T20:45:26.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3adffff640 1 -- 172.21.15.122:0/1623684639 learned_addr learned my addr 172.21.15.122:0/1623684639 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:26.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3ae5164640 1 --2- 172.21.15.122:0/1623684639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ae013b870 0x7f3ae013bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3ae4963640 1 --2- 172.21.15.122:0/1623684639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ac8004d10 0x7f3ae00b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3adffff640 1 -- 172.21.15.122:0/1623684639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ac8004d10 msgr2=0x7f3ae00b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:26.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3adffff640 1 --2- 172.21.15.122:0/1623684639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ac8004d10 0x7f3ae00b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3adffff640 1 -- 172.21.15.122:0/1623684639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ae013b870 msgr2=0x7f3ae013bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:26.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3adffff640 1 --2- 172.21.15.122:0/1623684639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ae013b870 0x7f3ae013bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:26.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3adffff640 1 -- 172.21.15.122:0/1623684639 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae01cd8b0 con 0x7f3ae01ce9b0 2024-03-16T20:45:26.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3ae5164640 1 --2- 172.21.15.122:0/1623684639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ae013b870 0x7f3ae013bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:26.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3adffff640 1 --2- 172.21.15.122:0/1623684639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ae01ce9b0 0x7f3ae01ddb30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000f790 tx=0x7f3ad000fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:26.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3add7fa640 1 -- 172.21.15.122:0/1623684639 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad000fea0 con 0x7f3ae01ce9b0 2024-03-16T20:45:26.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3add7fa640 1 -- 172.21.15.122:0/1623684639 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad000c8c0 con 0x7f3ae01ce9b0 2024-03-16T20:45:26.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3ae6bc7640 1 -- 172.21.15.122:0/1623684639 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae00a8c20 con 0x7f3ae01ce9b0 2024-03-16T20:45:26.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3add7fa640 1 -- 172.21.15.122:0/1623684639 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad000cc50 con 0x7f3ae01ce9b0 2024-03-16T20:45:26.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3ae6bc7640 1 -- 172.21.15.122:0/1623684639 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae013c230 con 0x7f3ae01ce9b0 2024-03-16T20:45:26.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3ae6bc7640 1 -- 172.21.15.122:0/1623684639 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae0075280 con 0x7f3ae01ce9b0 2024-03-16T20:45:26.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3add7fa640 1 -- 172.21.15.122:0/1623684639 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f3ad0020430 con 0x7f3ae01ce9b0 2024-03-16T20:45:26.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3add7fa640 1 --2- 172.21.15.122:0/1623684639 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3acc066e40 0x7f3acc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:26.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3add7fa640 1 -- 172.21.15.122:0/1623684639 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ad0087a50 con 0x7f3ae01ce9b0 2024-03-16T20:45:26.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3ae4963640 1 --2- 172.21.15.122:0/1623684639 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3acc066e40 0x7f3acc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:26.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.608+0000 7f3ae4963640 1 --2- 172.21.15.122:0/1623684639 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3acc066e40 0x7f3acc069300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4015f40 tx=0x7f3ad4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:26.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.612+0000 7f3add7fa640 1 -- 172.21.15.122:0/1623684639 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3ad0058460 con 0x7f3ae01ce9b0 2024-03-16T20:45:26.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:26.824+0000 7f3ae6bc7640 1 -- 172.21.15.122:0/1623684639 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]} v 0) v1 -- 0x7f3ae0071f30 con 0x7f3ae01ce9b0 2024-03-16T20:45:27.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.660+0000 7f3add7fa640 1 -- 172.21.15.122:0/1623684639 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]}]=0 v47) v1 ==== 137+0+0 (secure 0 0 0) 0x7f3ad0001560 con 0x7f3ae01ce9b0 2024-03-16T20:45:27.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.664+0000 7f3ae6bc7640 1 -- 172.21.15.122:0/1623684639 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3acc066e40 msgr2=0x7f3acc069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:27.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.664+0000 7f3ae6bc7640 1 --2- 172.21.15.122:0/1623684639 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3acc066e40 0x7f3acc069300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3ad4015f40 tx=0x7f3ad4008040 comp rx=0 tx=0).stop 2024-03-16T20:45:27.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.664+0000 7f3ae6bc7640 1 -- 172.21.15.122:0/1623684639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ae01ce9b0 msgr2=0x7f3ae01ddb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:27.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.664+0000 7f3ae6bc7640 1 --2- 172.21.15.122:0/1623684639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ae01ce9b0 0x7f3ae01ddb30 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3ad000f790 tx=0x7f3ad000fc50 comp rx=0 tx=0).stop 2024-03-16T20:45:27.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.664+0000 7f3ae6bc7640 1 -- 172.21.15.122:0/1623684639 shutdown_connections 2024-03-16T20:45:27.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.664+0000 7f3ae6bc7640 1 --2- 172.21.15.122:0/1623684639 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3acc066e40 0x7f3acc069300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:27.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.664+0000 7f3ae6bc7640 1 --2- 172.21.15.122:0/1623684639 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ae013b870 0x7f3ae013bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:27.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.664+0000 7f3ae6bc7640 1 --2- 172.21.15.122:0/1623684639 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ae01ce9b0 0x7f3ae01ddb30 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:27.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.664+0000 7f3ae6bc7640 1 --2- 172.21.15.122:0/1623684639 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ac8004d10 0x7f3ae00b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:27.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.664+0000 7f3ae6bc7640 1 -- 172.21.15.122:0/1623684639 >> 172.21.15.122:0/1623684639 conn(0x7f3ae00b8860 msgr2=0x7f3ae006af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:27.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.664+0000 7f3ae6bc7640 1 -- 172.21.15.122:0/1623684639 shutdown_connections 2024-03-16T20:45:27.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.664+0000 7f3ae6bc7640 1 -- 172.21.15.122:0/1623684639 wait complete. 2024-03-16T20:45:27.692 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-03-16T20:45:27.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.812+0000 7f381f9b9640 1 Processor -- start 2024-03-16T20:45:27.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f381f9b9640 1 -- start start 2024-03-16T20:45:27.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f381f9b9640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f381809ae30 0x7f38180af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:27.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f381f9b9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38180b6f00 0x7f381814e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:27.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f381f9b9640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38180b38e0 con 0x7f38180b4020 2024-03-16T20:45:27.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f381f9b9640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f381809b210 con 0x7f381809ae30 2024-03-16T20:45:27.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f381f9b9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38180b4fb0 con 0x7f38180b6f00 2024-03-16T20:45:27.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f381d755640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f381809ae30 0x7f38180af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:27.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f381d755640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f381809ae30 0x7f38180af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51218/0 (socket says 172.21.15.122:51218) 2024-03-16T20:45:27.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f381d755640 1 -- 172.21.15.122:0/3506153008 learned_addr learned my addr 172.21.15.122:0/3506153008 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:27.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f381cf54640 1 --1- 172.21.15.122:0/3506153008 >> v1:172.21.15.122:6789/0 conn(0x7f38180b4020 0x7f38180b67f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:51852/0 (socket says 172.21.15.122:51852) 2024-03-16T20:45:27.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f381df56640 1 --2- 172.21.15.122:0/3506153008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38180b6f00 0x7f381814e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:27.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f381d755640 1 -- 172.21.15.122:0/3506153008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38180b6f00 msgr2=0x7f381814e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:27.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f381d755640 1 --2- 172.21.15.122:0/3506153008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38180b6f00 0x7f381814e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:27.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f381d755640 1 -- 172.21.15.122:0/3506153008 >> v1:172.21.15.122:6789/0 conn(0x7f38180b4020 legacy=0x7f38180b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:27.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f381d755640 1 -- 172.21.15.122:0/3506153008 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38180b3290 con 0x7f381809ae30 2024-03-16T20:45:27.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f380ffff640 1 -- 172.21.15.122:0/3506153008 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065364980 0 0) 0x7f38180b38e0 con 0x7f38180b4020 2024-03-16T20:45:27.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f381d755640 1 --2- 172.21.15.122:0/3506153008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f381809ae30 0x7f38180af080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3814009870 tx=0x7f381401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac33061f62e4a16e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:27.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f380ffff640 1 -- 172.21.15.122:0/3506153008 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38140ac620 con 0x7f381809ae30 2024-03-16T20:45:27.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f380ffff640 1 -- 172.21.15.122:0/3506153008 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38140ac780 con 0x7f381809ae30 2024-03-16T20:45:27.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.816+0000 7f380ffff640 1 -- 172.21.15.122:0/3506153008 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38140acb10 con 0x7f381809ae30 2024-03-16T20:45:27.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 -- 172.21.15.122:0/3506153008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f381809ae30 msgr2=0x7f38180af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:27.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 --2- 172.21.15.122:0/3506153008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f381809ae30 0x7f38180af080 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f3814009870 tx=0x7f381401c4a0 comp rx=0 tx=0).stop 2024-03-16T20:45:27.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 -- 172.21.15.122:0/3506153008 shutdown_connections 2024-03-16T20:45:27.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 --2- 172.21.15.122:0/3506153008 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38180b6f00 0x7f381814e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:27.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 --2- 172.21.15.122:0/3506153008 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f381809ae30 0x7f38180af080 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:27.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 -- 172.21.15.122:0/3506153008 >> 172.21.15.122:0/3506153008 conn(0x7f38180b8d10 msgr2=0x7f381809a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:27.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 -- 172.21.15.122:0/3506153008 shutdown_connections 2024-03-16T20:45:27.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 -- 172.21.15.122:0/3506153008 wait complete. 2024-03-16T20:45:27.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 Processor -- start 2024-03-16T20:45:27.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 -- start start 2024-03-16T20:45:27.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f381809ae30 0x7f38180b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:27.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38180b4020 0x7f381814cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:27.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38180b6f00 0x7f381814d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:27.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381814dff0 con 0x7f38180b6f00 2024-03-16T20:45:27.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38180b0450 con 0x7f381809ae30 2024-03-16T20:45:27.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38180b2860 con 0x7f38180b4020 2024-03-16T20:45:27.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381df56640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38180b6f00 0x7f381814d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:27.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381df56640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38180b6f00 0x7f381814d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:51234/0 (socket says 172.21.15.122:51234) 2024-03-16T20:45:27.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381df56640 1 -- 172.21.15.122:0/4167933076 learned_addr learned my addr 172.21.15.122:0/4167933076 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:27.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381cf54640 1 --2- 172.21.15.122:0/4167933076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38180b4020 0x7f381814cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:27.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381df56640 1 -- 172.21.15.122:0/4167933076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38180b4020 msgr2=0x7f381814cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:27.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381df56640 1 --2- 172.21.15.122:0/4167933076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38180b4020 0x7f381814cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:27.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381df56640 1 -- 172.21.15.122:0/4167933076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f381809ae30 msgr2=0x7f38180b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:27.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381df56640 1 --2- 172.21.15.122:0/4167933076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f381809ae30 0x7f38180b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:27.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381df56640 1 -- 172.21.15.122:0/4167933076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38140094d0 con 0x7f38180b6f00 2024-03-16T20:45:27.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381df56640 1 --2- 172.21.15.122:0/4167933076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38180b6f00 0x7f381814d0f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f381000cc30 tx=0x7f381000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:27.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f380dffb640 1 -- 172.21.15.122:0/4167933076 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381000ce40 con 0x7f38180b6f00 2024-03-16T20:45:27.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f380dffb640 1 -- 172.21.15.122:0/4167933076 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38100039a0 con 0x7f38180b6f00 2024-03-16T20:45:27.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 -- 172.21.15.122:0/4167933076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38180b2b40 con 0x7f38180b6f00 2024-03-16T20:45:27.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f380dffb640 1 -- 172.21.15.122:0/4167933076 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810003d30 con 0x7f38180b6f00 2024-03-16T20:45:27.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 -- 172.21.15.122:0/4167933076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38180b09d0 con 0x7f38180b6f00 2024-03-16T20:45:27.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.820+0000 7f381f9b9640 1 -- 172.21.15.122:0/4167933076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381814c8d0 con 0x7f38180b6f00 2024-03-16T20:45:27.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.824+0000 7f380dffb640 1 -- 172.21.15.122:0/4167933076 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f3810003e90 con 0x7f38180b6f00 2024-03-16T20:45:27.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.824+0000 7f380dffb640 1 --2- 172.21.15.122:0/4167933076 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f37fc066e40 0x7f37fc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:27.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.824+0000 7f380dffb640 1 -- 172.21.15.122:0/4167933076 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3810084e50 con 0x7f38180b6f00 2024-03-16T20:45:27.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.824+0000 7f381d755640 1 --2- 172.21.15.122:0/4167933076 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f37fc066e40 0x7f37fc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:27.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.824+0000 7f381d755640 1 --2- 172.21.15.122:0/4167933076 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f37fc066e40 0x7f37fc069300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f38140ae000 tx=0x7f3814008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:27.831 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.828+0000 7f380dffb640 1 -- 172.21.15.122:0/4167933076 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f381814c8d0 con 0x7f38180b6f00 2024-03-16T20:45:27.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.988+0000 7f381f9b9640 1 -- 172.21.15.122:0/4167933076 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f381814c8d0 con 0x7f38180b6f00 2024-03-16T20:45:27.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.992+0000 7f380dffb640 1 -- 172.21.15.122:0/4167933076 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f381814c8d0 con 0x7f38180b6f00 2024-03-16T20:45:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.992+0000 7f381f9b9640 1 -- 172.21.15.122:0/4167933076 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f37fc066e40 msgr2=0x7f37fc069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.992+0000 7f381f9b9640 1 --2- 172.21.15.122:0/4167933076 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f37fc066e40 0x7f37fc069300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f38140ae000 tx=0x7f3814008040 comp rx=0 tx=0).stop 2024-03-16T20:45:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.992+0000 7f381f9b9640 1 -- 172.21.15.122:0/4167933076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38180b6f00 msgr2=0x7f381814d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.992+0000 7f381f9b9640 1 --2- 172.21.15.122:0/4167933076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38180b6f00 0x7f381814d0f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f381000cc30 tx=0x7f381000a5c0 comp rx=0 tx=0).stop 2024-03-16T20:45:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.992+0000 7f381f9b9640 1 -- 172.21.15.122:0/4167933076 shutdown_connections 2024-03-16T20:45:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.992+0000 7f381f9b9640 1 --2- 172.21.15.122:0/4167933076 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f37fc066e40 0x7f37fc069300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.992+0000 7f381f9b9640 1 --2- 172.21.15.122:0/4167933076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38180b6f00 0x7f381814d0f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.992+0000 7f381f9b9640 1 --2- 172.21.15.122:0/4167933076 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38180b4020 0x7f381814cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.992+0000 7f381f9b9640 1 --2- 172.21.15.122:0/4167933076 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f381809ae30 0x7f38180b1570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.992+0000 7f381f9b9640 1 -- 172.21.15.122:0/4167933076 >> 172.21.15.122:0/4167933076 conn(0x7f38180b8d10 msgr2=0x7f38180b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.992+0000 7f381f9b9640 1 -- 172.21.15.122:0/4167933076 shutdown_connections 2024-03-16T20:45:27.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:27.992+0000 7f381f9b9640 1 -- 172.21.15.122:0/4167933076 wait complete. 2024-03-16T20:45:28.014 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-16T20:45:28.015 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-16T20:45:28.063 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-16T20:45:28.122 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-16T20:45:28.122 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:45:28.122 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T20:45:28.127 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:45:28.127 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T20:45:28.175 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:28.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.340+0000 7f544a3f6640 1 Processor -- start 2024-03-16T20:45:28.342 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.340+0000 7f544a3f6640 1 -- start start 2024-03-16T20:45:28.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.340+0000 7f544a3f6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f54440b4400 0x7f5444140da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.340+0000 7f544a3f6640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f54440b4860 0x7f54441412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.340+0000 7f544a3f6640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f544406fd90 con 0x7f544406b1f0 2024-03-16T20:45:28.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.340+0000 7f544a3f6640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54440af860 con 0x7f54440b4860 2024-03-16T20:45:28.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.340+0000 7f544a3f6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f544406b5d0 con 0x7f54440b4400 2024-03-16T20:45:28.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.340+0000 7f5448993640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f54440b4860 0x7f54441412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.340+0000 7f5448993640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f54440b4860 0x7f54441412e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33982/0 (socket says 172.21.15.122:33982) 2024-03-16T20:45:28.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.340+0000 7f5448993640 1 -- 172.21.15.122:0/1426060680 learned_addr learned my addr 172.21.15.122:0/1426060680 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:28.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.340+0000 7f54437fe640 1 --2- 172.21.15.122:0/1426060680 >> v2:172.21.15.195:3300/0 conn(0x7f54440b4400 0x7f5444140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.340+0000 7f5448993640 1 -- 172.21.15.122:0/1426060680 >> v2:172.21.15.195:3300/0 conn(0x7f54440b4400 msgr2=0x7f5444140da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:28.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.340+0000 7f5448993640 1 --2- 172.21.15.122:0/1426060680 >> v2:172.21.15.195:3300/0 conn(0x7f54440b4400 0x7f5444140da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.340+0000 7f5448993640 1 -- 172.21.15.122:0/1426060680 >> v1:172.21.15.195:6789/0 conn(0x7f544406b1f0 legacy=0x7f54440af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:28.345 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.340+0000 7f5448993640 1 -- 172.21.15.122:0/1426060680 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5444070e10 con 0x7f54440b4860 2024-03-16T20:45:28.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f5448993640 1 --2- 172.21.15.122:0/1426060680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54440b4860 0x7f54441412e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f544406e480 tx=0x7f543c01c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1164440ed5cfacd4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:28.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f5442ffd640 1 -- 172.21.15.122:0/1426060680 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c0a5720 con 0x7f54440b4860 2024-03-16T20:45:28.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f5442ffd640 1 -- 172.21.15.122:0/1426060680 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543c0a5880 con 0x7f54440b4860 2024-03-16T20:45:28.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f5442ffd640 1 -- 172.21.15.122:0/1426060680 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c0a5c10 con 0x7f54440b4860 2024-03-16T20:45:28.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 -- 172.21.15.122:0/1426060680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54440b4860 msgr2=0x7f54441412e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:28.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 --2- 172.21.15.122:0/1426060680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54440b4860 0x7f54441412e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f544406e480 tx=0x7f543c01c860 comp rx=0 tx=0).stop 2024-03-16T20:45:28.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 -- 172.21.15.122:0/1426060680 shutdown_connections 2024-03-16T20:45:28.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 --2- 172.21.15.122:0/1426060680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54440b4860 0x7f54441412e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 --2- 172.21.15.122:0/1426060680 >> v2:172.21.15.195:3300/0 conn(0x7f54440b4400 0x7f5444140da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.347 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 -- 172.21.15.122:0/1426060680 >> 172.21.15.122:0/1426060680 conn(0x7f54440b8860 msgr2=0x7f54440b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:28.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 -- 172.21.15.122:0/1426060680 shutdown_connections 2024-03-16T20:45:28.348 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 -- 172.21.15.122:0/1426060680 wait complete. 2024-03-16T20:45:28.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 Processor -- start 2024-03-16T20:45:28.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 -- start start 2024-03-16T20:45:28.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f544406b1f0 0x7f5444146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54440b4400 0x7f54441309f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54440b4860 0x7f54441e2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54440b0050 con 0x7f544406b1f0 2024-03-16T20:45:28.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54440adaf0 con 0x7f54440b4400 2024-03-16T20:45:28.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544413fe90 con 0x7f54440b4860 2024-03-16T20:45:28.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f5443fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f544406b1f0 0x7f5444146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f5443fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f544406b1f0 0x7f5444146830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:33984/0 (socket says 172.21.15.122:33984) 2024-03-16T20:45:28.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f5443fff640 1 -- 172.21.15.122:0/2721758934 learned_addr learned my addr 172.21.15.122:0/2721758934 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:28.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f54437fe640 1 --2- 172.21.15.122:0/2721758934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54440b4400 0x7f54441309f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f5448993640 1 --2- 172.21.15.122:0/2721758934 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54440b4860 0x7f54441e2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f5443fff640 1 -- 172.21.15.122:0/2721758934 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54440b4860 msgr2=0x7f54441e2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:28.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f5443fff640 1 --2- 172.21.15.122:0/2721758934 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54440b4860 0x7f54441e2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f5443fff640 1 -- 172.21.15.122:0/2721758934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54440b4400 msgr2=0x7f54441309f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:28.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f5443fff640 1 --2- 172.21.15.122:0/2721758934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54440b4400 0x7f54441309f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f5443fff640 1 -- 172.21.15.122:0/2721758934 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c009b30 con 0x7f544406b1f0 2024-03-16T20:45:28.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f5443fff640 1 --2- 172.21.15.122:0/2721758934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f544406b1f0 0x7f5444146830 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f543400c450 tx=0x7f543400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:28.351 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f5440ff9640 1 -- 172.21.15.122:0/2721758934 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5434012830 con 0x7f544406b1f0 2024-03-16T20:45:28.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f5440ff9640 1 -- 172.21.15.122:0/2721758934 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54340129b0 con 0x7f544406b1f0 2024-03-16T20:45:28.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 -- 172.21.15.122:0/2721758934 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54441e2640 con 0x7f544406b1f0 2024-03-16T20:45:28.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f5440ff9640 1 -- 172.21.15.122:0/2721758934 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5434012d60 con 0x7f544406b1f0 2024-03-16T20:45:28.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.344+0000 7f544a3f6640 1 -- 172.21.15.122:0/2721758934 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5444130f30 con 0x7f544406b1f0 2024-03-16T20:45:28.352 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.348+0000 7f544a3f6640 1 -- 172.21.15.122:0/2721758934 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f544413fcf0 con 0x7f544406b1f0 2024-03-16T20:45:28.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.348+0000 7f5440ff9640 1 -- 172.21.15.122:0/2721758934 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f543401c460 con 0x7f544406b1f0 2024-03-16T20:45:28.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.348+0000 7f5440ff9640 1 --2- 172.21.15.122:0/2721758934 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5430066e40 0x7f5430069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.348+0000 7f5440ff9640 1 -- 172.21.15.122:0/2721758934 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5434084850 con 0x7f544406b1f0 2024-03-16T20:45:28.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.348+0000 7f54437fe640 1 --2- 172.21.15.122:0/2721758934 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5430066e40 0x7f5430069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.353 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.348+0000 7f54437fe640 1 --2- 172.21.15.122:0/2721758934 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5430066e40 0x7f5430069300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5444140970 tx=0x7f5438008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:28.355 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.352+0000 7f5440ff9640 1 -- 172.21.15.122:0/2721758934 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f544413fcf0 con 0x7f544406b1f0 2024-03-16T20:45:28.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.516+0000 7f544a3f6640 1 -- 172.21.15.122:0/2721758934 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f544413fcf0 con 0x7f544406b1f0 2024-03-16T20:45:28.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.520+0000 7f5440ff9640 1 -- 172.21.15.122:0/2721758934 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f544413fcf0 con 0x7f544406b1f0 2024-03-16T20:45:28.522 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:28.522 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:06.404479+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:28.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.520+0000 7f544a3f6640 1 -- 172.21.15.122:0/2721758934 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5430066e40 msgr2=0x7f5430069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:28.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.520+0000 7f544a3f6640 1 --2- 172.21.15.122:0/2721758934 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5430066e40 0x7f5430069300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5444140970 tx=0x7f5438008040 comp rx=0 tx=0).stop 2024-03-16T20:45:28.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.520+0000 7f544a3f6640 1 -- 172.21.15.122:0/2721758934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f544406b1f0 msgr2=0x7f5444146830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:28.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.520+0000 7f544a3f6640 1 --2- 172.21.15.122:0/2721758934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f544406b1f0 0x7f5444146830 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f543400c450 tx=0x7f543400c910 comp rx=0 tx=0).stop 2024-03-16T20:45:28.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.520+0000 7f544a3f6640 1 -- 172.21.15.122:0/2721758934 shutdown_connections 2024-03-16T20:45:28.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.520+0000 7f544a3f6640 1 --2- 172.21.15.122:0/2721758934 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5430066e40 0x7f5430069300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.520+0000 7f544a3f6640 1 --2- 172.21.15.122:0/2721758934 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54440b4860 0x7f54441e2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.520+0000 7f544a3f6640 1 --2- 172.21.15.122:0/2721758934 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54440b4400 0x7f54441309f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.520+0000 7f544a3f6640 1 --2- 172.21.15.122:0/2721758934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f544406b1f0 0x7f5444146830 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.520+0000 7f544a3f6640 1 -- 172.21.15.122:0/2721758934 >> 172.21.15.122:0/2721758934 conn(0x7f54440b8860 msgr2=0x7f54440b54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:28.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.520+0000 7f544a3f6640 1 -- 172.21.15.122:0/2721758934 shutdown_connections 2024-03-16T20:45:28.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.520+0000 7f544a3f6640 1 -- 172.21.15.122:0/2721758934 wait complete. 2024-03-16T20:45:28.541 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T20:44:52.653523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T20:45:02.758515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T20:45:03.710783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T20:45:28.542 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T20:45:28.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.660+0000 7f557c7cd640 1 Processor -- start 2024-03-16T20:45:28.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.660+0000 7f557c7cd640 1 -- start start 2024-03-16T20:45:28.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.660+0000 7f557c7cd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f557409ae30 0x7f55740af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.660+0000 7f557c7cd640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55740b38e0 con 0x7f55740b4020 2024-03-16T20:45:28.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.660+0000 7f557c7cd640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f557409b210 con 0x7f55740b6f00 2024-03-16T20:45:28.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.660+0000 7f557c7cd640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55740b4fb0 con 0x7f557409ae30 2024-03-16T20:45:28.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.660+0000 7f557a569640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f557409ae30 0x7f55740af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.660+0000 7f557a569640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f557409ae30 0x7f55740af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34000/0 (socket says 172.21.15.122:34000) 2024-03-16T20:45:28.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.660+0000 7f557a569640 1 -- 172.21.15.122:0/2662681656 learned_addr learned my addr 172.21.15.122:0/2662681656 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:28.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.660+0000 7f557a569640 1 -- 172.21.15.122:0/2662681656 >> v1:172.21.15.195:6789/0 conn(0x7f55740b6f00 legacy=0x7f557414e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:28.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.660+0000 7f557a569640 1 -- 172.21.15.122:0/2662681656 >> v1:172.21.15.122:6789/0 conn(0x7f55740b4020 legacy=0x7f55740b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:28.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.660+0000 7f557a569640 1 -- 172.21.15.122:0/2662681656 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55740b3290 con 0x7f557409ae30 2024-03-16T20:45:28.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.664+0000 7f557a569640 1 --2- 172.21.15.122:0/2662681656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f557409ae30 0x7f55740af080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5570009870 tx=0x7f557001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d19185970b704a1a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:28.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.664+0000 7f5579567640 1 -- 172.21.15.122:0/2662681656 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55700ab680 con 0x7f557409ae30 2024-03-16T20:45:28.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.664+0000 7f5579567640 1 -- 172.21.15.122:0/2662681656 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55700ab7e0 con 0x7f557409ae30 2024-03-16T20:45:28.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.664+0000 7f5579567640 1 -- 172.21.15.122:0/2662681656 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55700abb70 con 0x7f557409ae30 2024-03-16T20:45:28.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.664+0000 7f557c7cd640 1 -- 172.21.15.122:0/2662681656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f557409ae30 msgr2=0x7f55740af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:28.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.664+0000 7f557c7cd640 1 --2- 172.21.15.122:0/2662681656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f557409ae30 0x7f55740af080 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f5570009870 tx=0x7f557001c460 comp rx=0 tx=0).stop 2024-03-16T20:45:28.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.664+0000 7f557c7cd640 1 -- 172.21.15.122:0/2662681656 shutdown_connections 2024-03-16T20:45:28.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.664+0000 7f557c7cd640 1 --2- 172.21.15.122:0/2662681656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f557409ae30 0x7f55740af080 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.664+0000 7f557c7cd640 1 -- 172.21.15.122:0/2662681656 >> 172.21.15.122:0/2662681656 conn(0x7f55740b8d10 msgr2=0x7f557409a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:28.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.664+0000 7f557c7cd640 1 -- 172.21.15.122:0/2662681656 shutdown_connections 2024-03-16T20:45:28.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.664+0000 7f557c7cd640 1 -- 172.21.15.122:0/2662681656 wait complete. 2024-03-16T20:45:28.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557c7cd640 1 Processor -- start 2024-03-16T20:45:28.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557c7cd640 1 -- start start 2024-03-16T20:45:28.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557c7cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f557409ae30 0x7f55740b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557c7cd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f55740b4020 0x7f5574148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557c7cd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55740b6f00 0x7f55740712d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557c7cd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55741ddb80 con 0x7f557409ae30 2024-03-16T20:45:28.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557c7cd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5574070e40 con 0x7f55740b6f00 2024-03-16T20:45:28.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557c7cd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5574148dd0 con 0x7f55740b4020 2024-03-16T20:45:28.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557a569640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f557409ae30 0x7f55740b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557a569640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f557409ae30 0x7f55740b1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34008/0 (socket says 172.21.15.122:34008) 2024-03-16T20:45:28.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557a569640 1 -- 172.21.15.122:0/634468405 learned_addr learned my addr 172.21.15.122:0/634468405 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:28.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557ad6a640 1 --2- 172.21.15.122:0/634468405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55740b6f00 0x7f55740712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f5579d68640 1 --2- 172.21.15.122:0/634468405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f55740b4020 0x7f5574148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557a569640 1 -- 172.21.15.122:0/634468405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f55740b4020 msgr2=0x7f5574148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:28.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557a569640 1 --2- 172.21.15.122:0/634468405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f55740b4020 0x7f5574148890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557a569640 1 -- 172.21.15.122:0/634468405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55740b6f00 msgr2=0x7f55740712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:28.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557a569640 1 --2- 172.21.15.122:0/634468405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55740b6f00 0x7f55740712d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557a569640 1 -- 172.21.15.122:0/634468405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55700094d0 con 0x7f557409ae30 2024-03-16T20:45:28.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557ad6a640 1 --2- 172.21.15.122:0/634468405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55740b6f00 0x7f55740712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:28.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557a569640 1 --2- 172.21.15.122:0/634468405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f557409ae30 0x7f55740b1f40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f55700ad040 tx=0x7f557001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:28.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f556affd640 1 -- 172.21.15.122:0/634468405 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55700b4800 con 0x7f557409ae30 2024-03-16T20:45:28.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f556affd640 1 -- 172.21.15.122:0/634468405 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55700b4980 con 0x7f557409ae30 2024-03-16T20:45:28.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557c7cd640 1 -- 172.21.15.122:0/634468405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55741490a0 con 0x7f557409ae30 2024-03-16T20:45:28.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f556affd640 1 -- 172.21.15.122:0/634468405 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55700b4d30 con 0x7f557409ae30 2024-03-16T20:45:28.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557c7cd640 1 -- 172.21.15.122:0/634468405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5574072740 con 0x7f557409ae30 2024-03-16T20:45:28.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f557c7cd640 1 -- 172.21.15.122:0/634468405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f557406d800 con 0x7f557409ae30 2024-03-16T20:45:28.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f556affd640 1 -- 172.21.15.122:0/634468405 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f55700aa460 con 0x7f557409ae30 2024-03-16T20:45:28.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f556affd640 1 --2- 172.21.15.122:0/634468405 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5558066e40 0x7f5558069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f556affd640 1 -- 172.21.15.122:0/634468405 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f557011fab0 con 0x7f557409ae30 2024-03-16T20:45:28.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.668+0000 7f5579d68640 1 --2- 172.21.15.122:0/634468405 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5558066e40 0x7f5558069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.672+0000 7f5579d68640 1 --2- 172.21.15.122:0/634468405 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5558066e40 0x7f5558069300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5564007fd0 tx=0x7f5564009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:28.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.672+0000 7f556affd640 1 -- 172.21.15.122:0/634468405 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f55700eca50 con 0x7f557409ae30 2024-03-16T20:45:28.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.836+0000 7f557c7cd640 1 -- 172.21.15.122:0/634468405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5574071d50 con 0x7f557409ae30 2024-03-16T20:45:28.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.836+0000 7f556affd640 1 -- 172.21.15.122:0/634468405 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f55700f0110 con 0x7f557409ae30 2024-03-16T20:45:28.840 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-03-16T20:45:28.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.836+0000 7f557c7cd640 1 -- 172.21.15.122:0/634468405 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5558066e40 msgr2=0x7f5558069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:28.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.836+0000 7f557c7cd640 1 --2- 172.21.15.122:0/634468405 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5558066e40 0x7f5558069300 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5564007fd0 tx=0x7f5564009040 comp rx=0 tx=0).stop 2024-03-16T20:45:28.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.836+0000 7f557c7cd640 1 -- 172.21.15.122:0/634468405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f557409ae30 msgr2=0x7f55740b1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:28.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.836+0000 7f557c7cd640 1 --2- 172.21.15.122:0/634468405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f557409ae30 0x7f55740b1f40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f55700ad040 tx=0x7f557001aca0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.840+0000 7f557c7cd640 1 -- 172.21.15.122:0/634468405 shutdown_connections 2024-03-16T20:45:28.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.840+0000 7f557c7cd640 1 --2- 172.21.15.122:0/634468405 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5558066e40 0x7f5558069300 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.840+0000 7f557c7cd640 1 --2- 172.21.15.122:0/634468405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f55740b6f00 0x7f55740712d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.840+0000 7f557c7cd640 1 --2- 172.21.15.122:0/634468405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f55740b4020 0x7f5574148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.840+0000 7f557c7cd640 1 --2- 172.21.15.122:0/634468405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f557409ae30 0x7f55740b1f40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.840+0000 7f557c7cd640 1 -- 172.21.15.122:0/634468405 >> 172.21.15.122:0/634468405 conn(0x7f55740b8d10 msgr2=0x7f5574098330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:28.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.840+0000 7f557c7cd640 1 -- 172.21.15.122:0/634468405 shutdown_connections 2024-03-16T20:45:28.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.840+0000 7f557c7cd640 1 -- 172.21.15.122:0/634468405 wait complete. 2024-03-16T20:45:28.859 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T20:45:28.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f0895b08640 1 Processor -- start 2024-03-16T20:45:28.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f0895b08640 1 -- start start 2024-03-16T20:45:28.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f0895b08640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08901429f0 0x7f08900b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f0895b08640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08900b3fa0 con 0x7f08900b82b0 2024-03-16T20:45:28.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f0895b08640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f089009b210 con 0x7f089009ae30 2024-03-16T20:45:28.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f0895b08640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08900b50a0 con 0x7f08901429f0 2024-03-16T20:45:28.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f0886ffd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f08900b82b0 0x7f08901422e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39788/0 (socket says 172.21.15.122:39788) 2024-03-16T20:45:28.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f0886ffd640 1 -- 172.21.15.122:0/3492881973 learned_addr learned my addr 172.21.15.122:0/3492881973 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:28.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088ffff640 1 --2- 172.21.15.122:0/3492881973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08901429f0 0x7f08900b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088effd640 1 -- 172.21.15.122:0/3492881973 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862261086 0 0) 0x7f08900b3fa0 con 0x7f08900b82b0 2024-03-16T20:45:28.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088effd640 1 -- 172.21.15.122:0/3492881973 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f087c003530 con 0x7f08900b82b0 2024-03-16T20:45:28.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088effd640 1 -- 172.21.15.122:0/3492881973 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 668482018 0 0) 0x7f087c003530 con 0x7f08900b82b0 2024-03-16T20:45:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088effd640 1 -- 172.21.15.122:0/3492881973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08901429f0 msgr2=0x7f08900b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088effd640 1 --2- 172.21.15.122:0/3492881973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08901429f0 0x7f08900b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088effd640 1 -- 172.21.15.122:0/3492881973 >> v1:172.21.15.195:6789/0 conn(0x7f089009ae30 legacy=0x7f08900b7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088effd640 1 -- 172.21.15.122:0/3492881973 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08900b4ae0 con 0x7f08900b82b0 2024-03-16T20:45:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088effd640 1 -- 172.21.15.122:0/3492881973 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f08780016a0 con 0x7f08900b82b0 2024-03-16T20:45:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088effd640 1 -- 172.21.15.122:0/3492881973 >> v1:172.21.15.122:6789/0 conn(0x7f08900b82b0 legacy=0x7f08901422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088effd640 1 --2- 172.21.15.122:0/3492881973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f087c0039b0 0x7f087c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088effd640 1 --2- 172.21.15.122:0/3492881973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08901429f0 0x7f087c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088effd640 1 --2- 172.21.15.122:0/3492881973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f087c004d10 0x7f087c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088effd640 1 -- 172.21.15.122:0/3492881973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08900b50a0 con 0x7f087c0039b0 2024-03-16T20:45:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088effd640 1 -- 172.21.15.122:0/3492881973 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c00b990 con 0x7f087c004d10 2024-03-16T20:45:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088effd640 1 -- 172.21.15.122:0/3492881973 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c00bb50 con 0x7f08901429f0 2024-03-16T20:45:28.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088f7fe640 1 --2- 172.21.15.122:0/3492881973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f087c0039b0 0x7f087c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f0886ffd640 1 --2- 172.21.15.122:0/3492881973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08901429f0 0x7f087c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088ffff640 1 --2- 172.21.15.122:0/3492881973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f087c004d10 0x7f087c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088f7fe640 1 -- 172.21.15.122:0/3492881973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08901429f0 msgr2=0x7f087c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088f7fe640 1 --2- 172.21.15.122:0/3492881973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08901429f0 0x7f087c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088f7fe640 1 -- 172.21.15.122:0/3492881973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f087c004d10 msgr2=0x7f087c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088f7fe640 1 --2- 172.21.15.122:0/3492881973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f087c004d10 0x7f087c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.976+0000 7f088f7fe640 1 -- 172.21.15.122:0/3492881973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c003530 con 0x7f087c0039b0 2024-03-16T20:45:28.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f088f7fe640 1 --2- 172.21.15.122:0/3492881973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f087c0039b0 0x7f087c003d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0880001910 tx=0x7f088001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c96fbddee8795e93 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:28.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f088effd640 1 -- 172.21.15.122:0/3492881973 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08800ab600 con 0x7f087c0039b0 2024-03-16T20:45:28.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f088effd640 1 -- 172.21.15.122:0/3492881973 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08800a7050 con 0x7f087c0039b0 2024-03-16T20:45:28.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f088effd640 1 -- 172.21.15.122:0/3492881973 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08800ab9c0 con 0x7f087c0039b0 2024-03-16T20:45:28.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0895b08640 1 -- 172.21.15.122:0/3492881973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f087c0039b0 msgr2=0x7f087c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:28.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0895b08640 1 --2- 172.21.15.122:0/3492881973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f087c0039b0 0x7f087c003d90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0880001910 tx=0x7f088001c4c0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f088ffff640 1 -- 172.21.15.122:0/3492881973 reap_dead start 2024-03-16T20:45:28.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0895b08640 1 -- 172.21.15.122:0/3492881973 shutdown_connections 2024-03-16T20:45:28.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0895b08640 1 -- 172.21.15.122:0/3492881973 >> 172.21.15.122:0/3492881973 conn(0x7f08900b8d10 msgr2=0x7f089009a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:28.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0895b08640 1 -- 172.21.15.122:0/3492881973 shutdown_connections 2024-03-16T20:45:28.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0895b08640 1 -- 172.21.15.122:0/3492881973 wait complete. 2024-03-16T20:45:28.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0895b08640 1 Processor -- start 2024-03-16T20:45:28.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0895b08640 1 -- start start 2024-03-16T20:45:28.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0895b08640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f087c0039b0 0x7f089013e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0895b08640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08901e1000 0x7f08901e1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0895b08640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08901446e0 0x7f0890144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0895b08640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0890139ff0 con 0x7f08901e1000 2024-03-16T20:45:28.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0895b08640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0890145df0 con 0x7f087c0039b0 2024-03-16T20:45:28.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0895b08640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0890145b10 con 0x7f08901446e0 2024-03-16T20:45:28.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0886ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08901e1000 0x7f08901e1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0886ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08901e1000 0x7f08901e1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34028/0 (socket says 172.21.15.122:34028) 2024-03-16T20:45:28.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0886ffd640 1 -- 172.21.15.122:0/3383289765 learned_addr learned my addr 172.21.15.122:0/3383289765 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:28.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f088f7fe640 1 --2- 172.21.15.122:0/3383289765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f087c0039b0 0x7f089013e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0886ffd640 1 -- 172.21.15.122:0/3383289765 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08901446e0 msgr2=0x7f0890144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:28.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0886ffd640 1 --2- 172.21.15.122:0/3383289765 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08901446e0 0x7f0890144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0886ffd640 1 -- 172.21.15.122:0/3383289765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f087c0039b0 msgr2=0x7f089013e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:28.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0886ffd640 1 --2- 172.21.15.122:0/3383289765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f087c0039b0 0x7f089013e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:28.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.980+0000 7f0886ffd640 1 -- 172.21.15.122:0/3383289765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08901e0ae0 con 0x7f08901e1000 2024-03-16T20:45:28.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.984+0000 7f0886ffd640 1 --2- 172.21.15.122:0/3383289765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08901e1000 0x7f08901e1480 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f087800e780 tx=0x7f087800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:28.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.984+0000 7f088cff9640 1 -- 172.21.15.122:0/3383289765 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800c5e0 con 0x7f08901e1000 2024-03-16T20:45:28.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.984+0000 7f088cff9640 1 -- 172.21.15.122:0/3383289765 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087800c760 con 0x7f08901e1000 2024-03-16T20:45:28.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.984+0000 7f0895b08640 1 -- 172.21.15.122:0/3383289765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f089013a1d0 con 0x7f08901e1000 2024-03-16T20:45:28.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.984+0000 7f088cff9640 1 -- 172.21.15.122:0/3383289765 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800cb10 con 0x7f08901e1000 2024-03-16T20:45:28.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.984+0000 7f0895b08640 1 -- 172.21.15.122:0/3383289765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0890145730 con 0x7f08901e1000 2024-03-16T20:45:28.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.984+0000 7f0895b08640 1 -- 172.21.15.122:0/3383289765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0890145950 con 0x7f08901e1000 2024-03-16T20:45:28.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.984+0000 7f088cff9640 1 -- 172.21.15.122:0/3383289765 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f087800cc70 con 0x7f08901e1000 2024-03-16T20:45:28.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.984+0000 7f088cff9640 1 --2- 172.21.15.122:0/3383289765 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0870066e70 0x7f0870069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:28.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.984+0000 7f088cff9640 1 -- 172.21.15.122:0/3383289765 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0878084460 con 0x7f08901e1000 2024-03-16T20:45:28.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.984+0000 7f088f7fe640 1 --2- 172.21.15.122:0/3383289765 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0870066e70 0x7f0870069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:28.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.984+0000 7f088f7fe640 1 --2- 172.21.15.122:0/3383289765 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0870066e70 0x7f0870069330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f08800ad000 tx=0x7f0880001330 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:28.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:28.988+0000 7f088cff9640 1 -- 172.21.15.122:0/3383289765 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f08780512e0 con 0x7f08901e1000 2024-03-16T20:45:29.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.140+0000 7f0895b08640 1 -- 172.21.15.122:0/3383289765 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0890074bb0 con 0x7f08901e1000 2024-03-16T20:45:29.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.144+0000 7f088cff9640 1 -- 172.21.15.122:0/3383289765 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f08780549a0 con 0x7f08901e1000 2024-03-16T20:45:29.147 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-03-16T20:45:29.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.144+0000 7f0895b08640 1 -- 172.21.15.122:0/3383289765 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0870066e70 msgr2=0x7f0870069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.144+0000 7f0895b08640 1 --2- 172.21.15.122:0/3383289765 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0870066e70 0x7f0870069330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f08800ad000 tx=0x7f0880001330 comp rx=0 tx=0).stop 2024-03-16T20:45:29.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.144+0000 7f0895b08640 1 -- 172.21.15.122:0/3383289765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08901e1000 msgr2=0x7f08901e1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.144+0000 7f0895b08640 1 --2- 172.21.15.122:0/3383289765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08901e1000 0x7f08901e1480 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f087800e780 tx=0x7f087800ec40 comp rx=0 tx=0).stop 2024-03-16T20:45:29.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.144+0000 7f0895b08640 1 -- 172.21.15.122:0/3383289765 shutdown_connections 2024-03-16T20:45:29.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.144+0000 7f0895b08640 1 --2- 172.21.15.122:0/3383289765 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0870066e70 0x7f0870069330 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.144+0000 7f0895b08640 1 --2- 172.21.15.122:0/3383289765 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f08901446e0 0x7f0890144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.144+0000 7f0895b08640 1 --2- 172.21.15.122:0/3383289765 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f08901e1000 0x7f08901e1480 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.144+0000 7f0895b08640 1 --2- 172.21.15.122:0/3383289765 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f087c0039b0 0x7f089013e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.144+0000 7f0895b08640 1 -- 172.21.15.122:0/3383289765 >> 172.21.15.122:0/3383289765 conn(0x7f08900b8d10 msgr2=0x7f08900972b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:29.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.144+0000 7f0895b08640 1 -- 172.21.15.122:0/3383289765 shutdown_connections 2024-03-16T20:45:29.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.144+0000 7f0895b08640 1 -- 172.21.15.122:0/3383289765 wait complete. 2024-03-16T20:45:29.166 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T20:45:29.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f810dc640 1 Processor -- start 2024-03-16T20:45:29.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f810dc640 1 -- start start 2024-03-16T20:45:29.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f810dc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8f7c09ae30 0x7f8f7c0b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f810dc640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f8f7c0b7ba0 0x7f8f7c0b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f810dc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f7c1422e0 0x7f8f7c142770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f810dc640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f7c0b50a0 con 0x7f8f7c0b7ba0 2024-03-16T20:45:29.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f810dc640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f7c0b8590 con 0x7f8f7c09ae30 2024-03-16T20:45:29.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f810dc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c0b3fa0 con 0x7f8f7c1422e0 2024-03-16T20:45:29.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f7259c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f8f7c0b7ba0 0x7f8f7c0b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f7259c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f8f7c0b7ba0 0x7f8f7c0b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34032/0 (socket says 172.21.15.122:34032) 2024-03-16T20:45:29.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f7259c640 1 -- 172.21.15.122:0/752110440 learned_addr learned my addr 172.21.15.122:0/752110440 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:29.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f7ad9d640 1 --2- 172.21.15.122:0/752110440 >> v2:172.21.15.195:3300/0 conn(0x7f8f7c09ae30 0x7f8f7c0b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f7b59e640 1 --2- 172.21.15.122:0/752110440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f7c1422e0 0x7f8f7c142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f7259c640 1 -- 172.21.15.122:0/752110440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f7c1422e0 msgr2=0x7f8f7c142770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:29.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f7259c640 1 --2- 172.21.15.122:0/752110440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f7c1422e0 0x7f8f7c142770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f7259c640 1 -- 172.21.15.122:0/752110440 >> v2:172.21.15.195:3300/0 conn(0x7f8f7c09ae30 msgr2=0x7f8f7c0b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:29.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f7259c640 1 --2- 172.21.15.122:0/752110440 >> v2:172.21.15.195:3300/0 conn(0x7f8f7c09ae30 0x7f8f7c0b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f7259c640 1 -- 172.21.15.122:0/752110440 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c142cb0 con 0x7f8f7c0b7ba0 2024-03-16T20:45:29.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f7259c640 1 --2- 172.21.15.122:0/752110440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c0b7ba0 0x7f8f7c0b8050 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c06e120 tx=0x7f8f6401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edfce61f34e02836 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:29.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f7a59c640 1 -- 172.21.15.122:0/752110440 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f640aa640 con 0x7f8f7c0b7ba0 2024-03-16T20:45:29.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f7a59c640 1 -- 172.21.15.122:0/752110440 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f640aa7a0 con 0x7f8f7c0b7ba0 2024-03-16T20:45:29.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.284+0000 7f8f7a59c640 1 -- 172.21.15.122:0/752110440 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f640aab30 con 0x7f8f7c0b7ba0 2024-03-16T20:45:29.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 -- 172.21.15.122:0/752110440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c0b7ba0 msgr2=0x7f8f7c0b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 --2- 172.21.15.122:0/752110440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c0b7ba0 0x7f8f7c0b8050 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c06e120 tx=0x7f8f6401c520 comp rx=0 tx=0).stop 2024-03-16T20:45:29.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 -- 172.21.15.122:0/752110440 shutdown_connections 2024-03-16T20:45:29.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 --2- 172.21.15.122:0/752110440 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f7c1422e0 0x7f8f7c142770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 --2- 172.21.15.122:0/752110440 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c0b7ba0 0x7f8f7c0b8050 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 --2- 172.21.15.122:0/752110440 >> v2:172.21.15.195:3300/0 conn(0x7f8f7c09ae30 0x7f8f7c0b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 -- 172.21.15.122:0/752110440 >> 172.21.15.122:0/752110440 conn(0x7f8f7c0b8d10 msgr2=0x7f8f7c09a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:29.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 -- 172.21.15.122:0/752110440 shutdown_connections 2024-03-16T20:45:29.291 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 -- 172.21.15.122:0/752110440 wait complete. 2024-03-16T20:45:29.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 Processor -- start 2024-03-16T20:45:29.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 -- start start 2024-03-16T20:45:29.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c09ae30 0x7f8f7c09ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f7c0b7ba0 0x7f8f7c0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f7c1422e0 0x7f8f7c1e2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c0af360 con 0x7f8f7c09ae30 2024-03-16T20:45:29.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c1e0220 con 0x7f8f7c0b7ba0 2024-03-16T20:45:29.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7c13bea0 con 0x7f8f7c1422e0 2024-03-16T20:45:29.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f7ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c09ae30 0x7f8f7c09ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f7ad9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c09ae30 0x7f8f7c09ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34048/0 (socket says 172.21.15.122:34048) 2024-03-16T20:45:29.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f7ad9d640 1 -- 172.21.15.122:0/3836254027 learned_addr learned my addr 172.21.15.122:0/3836254027 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:29.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f7259c640 1 --2- 172.21.15.122:0/3836254027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f7c0b7ba0 0x7f8f7c0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f7b59e640 1 --2- 172.21.15.122:0/3836254027 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f7c1422e0 0x7f8f7c1e2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f7ad9d640 1 -- 172.21.15.122:0/3836254027 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f7c1422e0 msgr2=0x7f8f7c1e2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f7ad9d640 1 --2- 172.21.15.122:0/3836254027 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f7c1422e0 0x7f8f7c1e2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f7ad9d640 1 -- 172.21.15.122:0/3836254027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f7c0b7ba0 msgr2=0x7f8f7c0ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f7ad9d640 1 --2- 172.21.15.122:0/3836254027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f7c0b7ba0 0x7f8f7c0ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f7ad9d640 1 -- 172.21.15.122:0/3836254027 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f640094d0 con 0x7f8f7c09ae30 2024-03-16T20:45:29.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f7b59e640 1 --2- 172.21.15.122:0/3836254027 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f7c1422e0 0x7f8f7c1e2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:29.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f7ad9d640 1 --2- 172.21.15.122:0/3836254027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c09ae30 0x7f8f7c09ce60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00d800 tx=0x7f8f6c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:29.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f73fff640 1 -- 172.21.15.122:0/3836254027 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c009640 con 0x7f8f7c09ae30 2024-03-16T20:45:29.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f73fff640 1 -- 172.21.15.122:0/3836254027 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6c0097a0 con 0x7f8f7c09ae30 2024-03-16T20:45:29.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f73fff640 1 -- 172.21.15.122:0/3836254027 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c009b10 con 0x7f8f7c09ae30 2024-03-16T20:45:29.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 -- 172.21.15.122:0/3836254027 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f7c1e06b0 con 0x7f8f7c09ae30 2024-03-16T20:45:29.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 -- 172.21.15.122:0/3836254027 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f7c143420 con 0x7f8f7c09ae30 2024-03-16T20:45:29.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.288+0000 7f8f810dc640 1 -- 172.21.15.122:0/3836254027 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f7c074bb0 con 0x7f8f7c09ae30 2024-03-16T20:45:29.296 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.292+0000 7f8f73fff640 1 -- 172.21.15.122:0/3836254027 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f8f6c009c70 con 0x7f8f7c09ae30 2024-03-16T20:45:29.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.292+0000 7f8f73fff640 1 --2- 172.21.15.122:0/3836254027 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8f5c066e40 0x7f8f5c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.292+0000 7f8f73fff640 1 -- 172.21.15.122:0/3836254027 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f6c085150 con 0x7f8f7c09ae30 2024-03-16T20:45:29.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.292+0000 7f8f7259c640 1 --2- 172.21.15.122:0/3836254027 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8f5c066e40 0x7f8f5c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.292+0000 7f8f7259c640 1 --2- 172.21.15.122:0/3836254027 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8f5c066e40 0x7f8f5c069300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8f640ae000 tx=0x7f8f640a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:29.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.296+0000 7f8f73fff640 1 -- 172.21.15.122:0/3836254027 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8f6c051fd0 con 0x7f8f7c09ae30 2024-03-16T20:45:29.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.456+0000 7f8f810dc640 1 -- 172.21.15.122:0/3836254027 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8f7c0b7720 con 0x7f8f7c09ae30 2024-03-16T20:45:29.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.456+0000 7f8f73fff640 1 -- 172.21.15.122:0/3836254027 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8f6c055690 con 0x7f8f7c09ae30 2024-03-16T20:45:29.460 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-03-16T20:45:29.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.456+0000 7f8f810dc640 1 -- 172.21.15.122:0/3836254027 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8f5c066e40 msgr2=0x7f8f5c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.456+0000 7f8f810dc640 1 --2- 172.21.15.122:0/3836254027 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8f5c066e40 0x7f8f5c069300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8f640ae000 tx=0x7f8f640a7000 comp rx=0 tx=0).stop 2024-03-16T20:45:29.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.456+0000 7f8f810dc640 1 -- 172.21.15.122:0/3836254027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c09ae30 msgr2=0x7f8f7c09ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.456+0000 7f8f810dc640 1 --2- 172.21.15.122:0/3836254027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c09ae30 0x7f8f7c09ce60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00d800 tx=0x7f8f6c00dcc0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.456+0000 7f8f810dc640 1 -- 172.21.15.122:0/3836254027 shutdown_connections 2024-03-16T20:45:29.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.456+0000 7f8f810dc640 1 --2- 172.21.15.122:0/3836254027 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8f5c066e40 0x7f8f5c069300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.456+0000 7f8f810dc640 1 --2- 172.21.15.122:0/3836254027 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8f7c1422e0 0x7f8f7c1e2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.456+0000 7f8f810dc640 1 --2- 172.21.15.122:0/3836254027 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8f7c0b7ba0 0x7f8f7c0ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.456+0000 7f8f810dc640 1 --2- 172.21.15.122:0/3836254027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8f7c09ae30 0x7f8f7c09ce60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.456+0000 7f8f810dc640 1 -- 172.21.15.122:0/3836254027 >> 172.21.15.122:0/3836254027 conn(0x7f8f7c0b8d10 msgr2=0x7f8f7c13e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:29.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.456+0000 7f8f810dc640 1 -- 172.21.15.122:0/3836254027 shutdown_connections 2024-03-16T20:45:29.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.456+0000 7f8f810dc640 1 -- 172.21.15.122:0/3836254027 wait complete. 2024-03-16T20:45:29.478 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T20:45:29.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.584+0000 7fc0092e7640 1 Processor -- start 2024-03-16T20:45:29.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.584+0000 7fc0092e7640 1 -- start start 2024-03-16T20:45:29.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.584+0000 7fc0092e7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0041429f0 0x7fc0040b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.584+0000 7fc0092e7640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0040b3fa0 con 0x7fc00409ae30 2024-03-16T20:45:29.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.584+0000 7fc0092e7640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc00409b210 con 0x7fc0040b82b0 2024-03-16T20:45:29.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.584+0000 7fc0092e7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040b50a0 con 0x7fc0041429f0 2024-03-16T20:45:29.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.584+0000 7fc002ffd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fc00409ae30 0x7fc0040b7ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39792/0 (socket says 172.21.15.122:39792) 2024-03-16T20:45:29.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc002ffd640 1 -- 172.21.15.122:0/2314063403 learned_addr learned my addr 172.21.15.122:0/2314063403 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:29.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc0037fe640 1 --2- 172.21.15.122:0/2314063403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0041429f0 0x7fc0040b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 -- 172.21.15.122:0/2314063403 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4021243228 0 0) 0x7fc0040b3fa0 con 0x7fc00409ae30 2024-03-16T20:45:29.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 -- 172.21.15.122:0/2314063403 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbff0003530 con 0x7fc00409ae30 2024-03-16T20:45:29.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 -- 172.21.15.122:0/2314063403 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507012264 0 0) 0x7fc00409b210 con 0x7fc0040b82b0 2024-03-16T20:45:29.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 -- 172.21.15.122:0/2314063403 --> v1:172.21.15.195:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0040b3fa0 con 0x7fc0040b82b0 2024-03-16T20:45:29.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 -- 172.21.15.122:0/2314063403 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 636906418 0 0) 0x7fbff0003530 con 0x7fc00409ae30 2024-03-16T20:45:29.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 -- 172.21.15.122:0/2314063403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0041429f0 msgr2=0x7fc0040b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:29.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 --2- 172.21.15.122:0/2314063403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0041429f0 0x7fc0040b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 -- 172.21.15.122:0/2314063403 >> v1:172.21.15.195:6789/0 conn(0x7fc0040b82b0 legacy=0x7fc0041422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 -- 172.21.15.122:0/2314063403 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0040b4ae0 con 0x7fc00409ae30 2024-03-16T20:45:29.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 -- 172.21.15.122:0/2314063403 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7fbff40016a0 con 0x7fc00409ae30 2024-03-16T20:45:29.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc0037fe640 1 --2- 172.21.15.122:0/2314063403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0041429f0 0x7fc0040b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:45:29.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 -- 172.21.15.122:0/2314063403 >> v1:172.21.15.122:6789/0 conn(0x7fc00409ae30 legacy=0x7fc0040b7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 --2- 172.21.15.122:0/2314063403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbff0003b40 0x7fbff0003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 --2- 172.21.15.122:0/2314063403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0041429f0 0x7fbff00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 --2- 172.21.15.122:0/2314063403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbff0004d10 0x7fbff0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 -- 172.21.15.122:0/2314063403 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040b50a0 con 0x7fbff0004d10 2024-03-16T20:45:29.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 -- 172.21.15.122:0/2314063403 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff000ba40 con 0x7fbff0003b40 2024-03-16T20:45:29.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 -- 172.21.15.122:0/2314063403 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff000bb80 con 0x7fc0041429f0 2024-03-16T20:45:29.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc0037fe640 1 --2- 172.21.15.122:0/2314063403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbff0004d10 0x7fbff0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc0027fc640 1 --2- 172.21.15.122:0/2314063403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0041429f0 0x7fbff00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc002ffd640 1 --2- 172.21.15.122:0/2314063403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbff0003b40 0x7fbff0003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc0037fe640 1 -- 172.21.15.122:0/2314063403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0041429f0 msgr2=0x7fbff00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:29.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc0037fe640 1 --2- 172.21.15.122:0/2314063403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0041429f0 0x7fbff00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc0037fe640 1 -- 172.21.15.122:0/2314063403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbff0003b40 msgr2=0x7fbff0003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:29.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc0037fe640 1 --2- 172.21.15.122:0/2314063403 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbff0003b40 0x7fbff0003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc0037fe640 1 -- 172.21.15.122:0/2314063403 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc009420 con 0x7fbff0004d10 2024-03-16T20:45:29.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc0027fc640 1 --2- 172.21.15.122:0/2314063403 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc0041429f0 0x7fbff00047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:29.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc0037fe640 1 --2- 172.21.15.122:0/2314063403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbff0004d10 0x7fbff0005190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009ce0 tx=0x7fbffc01cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fdb88020c0efa8e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:29.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 -- 172.21.15.122:0/2314063403 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0a57a0 con 0x7fbff0004d10 2024-03-16T20:45:29.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 -- 172.21.15.122:0/2314063403 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbffc0a5900 con 0x7fbff0004d10 2024-03-16T20:45:29.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.588+0000 7fc001ffb640 1 -- 172.21.15.122:0/2314063403 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0a5cb0 con 0x7fbff0004d10 2024-03-16T20:45:29.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0092e7640 1 -- 172.21.15.122:0/2314063403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbff0004d10 msgr2=0x7fbff0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0092e7640 1 --2- 172.21.15.122:0/2314063403 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbff0004d10 0x7fbff0005190 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009ce0 tx=0x7fbffc01cbf0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0037fe640 1 -- 172.21.15.122:0/2314063403 reap_dead start 2024-03-16T20:45:29.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0092e7640 1 -- 172.21.15.122:0/2314063403 shutdown_connections 2024-03-16T20:45:29.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0092e7640 1 -- 172.21.15.122:0/2314063403 >> 172.21.15.122:0/2314063403 conn(0x7fc0040b8d10 msgr2=0x7fc00409a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:29.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0092e7640 1 -- 172.21.15.122:0/2314063403 shutdown_connections 2024-03-16T20:45:29.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0092e7640 1 -- 172.21.15.122:0/2314063403 wait complete. 2024-03-16T20:45:29.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0092e7640 1 Processor -- start 2024-03-16T20:45:29.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0092e7640 1 -- start start 2024-03-16T20:45:29.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0092e7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0040b1e80 0x7fc004142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0092e7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc004143fd0 0x7fc0041433e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0092e7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc00413de50 0x7fc00413e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0092e7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0041444b0 con 0x7fc0040b1e80 2024-03-16T20:45:29.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0092e7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040b2260 con 0x7fc004143fd0 2024-03-16T20:45:29.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0092e7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0041455c0 con 0x7fc00413de50 2024-03-16T20:45:29.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc002ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0040b1e80 0x7fc004142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc002ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0040b1e80 0x7fc004142ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34074/0 (socket says 172.21.15.122:34074) 2024-03-16T20:45:29.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc002ffd640 1 -- 172.21.15.122:0/1867173013 learned_addr learned my addr 172.21.15.122:0/1867173013 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:29.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0037fe640 1 --2- 172.21.15.122:0/1867173013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc00413de50 0x7fc00413e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0027fc640 1 --2- 172.21.15.122:0/1867173013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc004143fd0 0x7fc0041433e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc002ffd640 1 -- 172.21.15.122:0/1867173013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc00413de50 msgr2=0x7fc00413e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc002ffd640 1 --2- 172.21.15.122:0/1867173013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc00413de50 0x7fc00413e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc002ffd640 1 -- 172.21.15.122:0/1867173013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc004143fd0 msgr2=0x7fc0041433e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc002ffd640 1 --2- 172.21.15.122:0/1867173013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc004143fd0 0x7fc0041433e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc002ffd640 1 -- 172.21.15.122:0/1867173013 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc004145c10 con 0x7fc0040b1e80 2024-03-16T20:45:29.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc002ffd640 1 --2- 172.21.15.122:0/1867173013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0040b1e80 0x7fc004142ea0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbff4004850 tx=0x7fbff4011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:29.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fbffb7fe640 1 -- 172.21.15.122:0/1867173013 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff400fb60 con 0x7fc0040b1e80 2024-03-16T20:45:29.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fbffb7fe640 1 -- 172.21.15.122:0/1867173013 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff400fd00 con 0x7fc0040b1e80 2024-03-16T20:45:29.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fbffb7fe640 1 -- 172.21.15.122:0/1867173013 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff4015650 con 0x7fc0040b1e80 2024-03-16T20:45:29.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0092e7640 1 -- 172.21.15.122:0/1867173013 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc004145ef0 con 0x7fc0040b1e80 2024-03-16T20:45:29.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0092e7640 1 -- 172.21.15.122:0/1867173013 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0040af3d0 con 0x7fc0040b1e80 2024-03-16T20:45:29.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.592+0000 7fc0092e7640 1 -- 172.21.15.122:0/1867173013 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc004146130 con 0x7fc0040b1e80 2024-03-16T20:45:29.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.596+0000 7fbffb7fe640 1 -- 172.21.15.122:0/1867173013 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fbff40157b0 con 0x7fc0040b1e80 2024-03-16T20:45:29.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.596+0000 7fbffb7fe640 1 --2- 172.21.15.122:0/1867173013 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fbfe4066e70 0x7fbfe4069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.596+0000 7fbffb7fe640 1 -- 172.21.15.122:0/1867173013 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbff4087bf0 con 0x7fc0040b1e80 2024-03-16T20:45:29.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.596+0000 7fc0027fc640 1 --2- 172.21.15.122:0/1867173013 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fbfe4066e70 0x7fbfe4069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.596+0000 7fc0027fc640 1 --2- 172.21.15.122:0/1867173013 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fbfe4066e70 0x7fbfe4069330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc004072710 tx=0x7fbfec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:29.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.600+0000 7fbffb7fe640 1 -- 172.21.15.122:0/1867173013 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc004146130 con 0x7fc0040b1e80 2024-03-16T20:45:29.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.772+0000 7fc0092e7640 1 -- 172.21.15.122:0/1867173013 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc004146130 con 0x7fc0040b1e80 2024-03-16T20:45:29.775 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.772+0000 7fbffb7fe640 1 -- 172.21.15.122:0/1867173013 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc004146130 con 0x7fc0040b1e80 2024-03-16T20:45:29.775 INFO:teuthology.orchestra.run.smithi122.stderr:listed 0 entries 2024-03-16T20:45:29.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.772+0000 7fc0092e7640 1 -- 172.21.15.122:0/1867173013 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fbfe4066e70 msgr2=0x7fbfe4069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.772+0000 7fc0092e7640 1 --2- 172.21.15.122:0/1867173013 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fbfe4066e70 0x7fbfe4069330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc004072710 tx=0x7fbfec009040 comp rx=0 tx=0).stop 2024-03-16T20:45:29.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.772+0000 7fc0092e7640 1 -- 172.21.15.122:0/1867173013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0040b1e80 msgr2=0x7fc004142ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.772+0000 7fc0092e7640 1 --2- 172.21.15.122:0/1867173013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0040b1e80 0x7fc004142ea0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fbff4004850 tx=0x7fbff4011c20 comp rx=0 tx=0).stop 2024-03-16T20:45:29.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.772+0000 7fc0092e7640 1 -- 172.21.15.122:0/1867173013 shutdown_connections 2024-03-16T20:45:29.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.772+0000 7fc0092e7640 1 --2- 172.21.15.122:0/1867173013 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fbfe4066e70 0x7fbfe4069330 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.772+0000 7fc0092e7640 1 --2- 172.21.15.122:0/1867173013 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc00413de50 0x7fc00413e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.772+0000 7fc0092e7640 1 --2- 172.21.15.122:0/1867173013 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc004143fd0 0x7fc0041433e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.772+0000 7fc0092e7640 1 --2- 172.21.15.122:0/1867173013 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc0040b1e80 0x7fc004142ea0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.772+0000 7fc0092e7640 1 -- 172.21.15.122:0/1867173013 >> 172.21.15.122:0/1867173013 conn(0x7fc0040b8d10 msgr2=0x7fc004074320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:29.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.772+0000 7fc0092e7640 1 -- 172.21.15.122:0/1867173013 shutdown_connections 2024-03-16T20:45:29.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.772+0000 7fc0092e7640 1 -- 172.21.15.122:0/1867173013 wait complete. 2024-03-16T20:45:29.793 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:45:29.793 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T20:45:29.799 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:29.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fc5566640 1 Processor -- start 2024-03-16T20:45:29.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fc5566640 1 -- start start 2024-03-16T20:45:29.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fc5566640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8fc0072120 0x7f8fc00afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fc5566640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc00b1050 con 0x7f8fc013ed60 2024-03-16T20:45:29.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fc5566640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc0072500 con 0x7f8fc0148b40 2024-03-16T20:45:29.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fc5566640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc00b38a0 con 0x7f8fc0072120 2024-03-16T20:45:29.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbe7fc640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f8fc013ed60 0x7f8fc0148430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39808/0 (socket says 172.21.15.122:39808) 2024-03-16T20:45:29.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbe7fc640 1 -- 172.21.15.122:0/504866887 learned_addr learned my addr 172.21.15.122:0/504866887 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:29.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbeffd640 1 --2- 172.21.15.122:0/504866887 >> v2:172.21.15.195:3300/0 conn(0x7f8fc0072120 0x7f8fc00afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbdffb640 1 -- 172.21.15.122:0/504866887 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825653450 0 0) 0x7f8fc00b1050 con 0x7f8fc013ed60 2024-03-16T20:45:29.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbdffb640 1 -- 172.21.15.122:0/504866887 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fa8003530 con 0x7f8fc013ed60 2024-03-16T20:45:29.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbdffb640 1 -- 172.21.15.122:0/504866887 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 979335554 0 0) 0x7f8fa8003530 con 0x7f8fc013ed60 2024-03-16T20:45:29.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbdffb640 1 -- 172.21.15.122:0/504866887 >> v2:172.21.15.195:3300/0 conn(0x7f8fc0072120 msgr2=0x7f8fc00afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:29.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbdffb640 1 --2- 172.21.15.122:0/504866887 >> v2:172.21.15.195:3300/0 conn(0x7f8fc0072120 0x7f8fc00afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbdffb640 1 -- 172.21.15.122:0/504866887 >> v1:172.21.15.195:6789/0 conn(0x7f8fc0148b40 legacy=0x7f8fc014f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbdffb640 1 -- 172.21.15.122:0/504866887 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00b3b80 con 0x7f8fc013ed60 2024-03-16T20:45:29.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbdffb640 1 -- 172.21.15.122:0/504866887 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f8fb0001680 con 0x7f8fc013ed60 2024-03-16T20:45:29.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbdffb640 1 -- 172.21.15.122:0/504866887 >> v1:172.21.15.122:6789/0 conn(0x7f8fc013ed60 legacy=0x7f8fc0148430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbdffb640 1 --2- 172.21.15.122:0/504866887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fa8003980 0x7f8fa8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbdffb640 1 --2- 172.21.15.122:0/504866887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fa80047d0 0x7f8fa8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbdffb640 1 --2- 172.21.15.122:0/504866887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fa8005190 0x7f8fa800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbdffb640 1 -- 172.21.15.122:0/504866887 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00b38a0 con 0x7f8fa8003980 2024-03-16T20:45:29.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbdffb640 1 -- 172.21.15.122:0/504866887 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa800c130 con 0x7f8fa80047d0 2024-03-16T20:45:29.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbdffb640 1 -- 172.21.15.122:0/504866887 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa800c2f0 con 0x7f8fa8005190 2024-03-16T20:45:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbeffd640 1 --2- 172.21.15.122:0/504866887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fa8003980 0x7f8fa8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbf7fe640 1 --2- 172.21.15.122:0/504866887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fa8005190 0x7f8fa800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbeffd640 1 -- 172.21.15.122:0/504866887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fa8005190 msgr2=0x7f8fa800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbeffd640 1 --2- 172.21.15.122:0/504866887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fa8005190 0x7f8fa800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbeffd640 1 -- 172.21.15.122:0/504866887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fa80047d0 msgr2=0x7f8fa8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbeffd640 1 --2- 172.21.15.122:0/504866887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fa80047d0 0x7f8fa8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbf7fe640 1 -- 172.21.15.122:0/504866887 reap_dead start 2024-03-16T20:45:29.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.960+0000 7f8fbeffd640 1 -- 172.21.15.122:0/504866887 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa8003530 con 0x7f8fa8003980 2024-03-16T20:45:29.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fbeffd640 1 --2- 172.21.15.122:0/504866887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fa8003980 0x7f8fa8003da0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009ac0 tx=0x7f8fb401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ee762a3d65dd75a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:29.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fbdffb640 1 -- 172.21.15.122:0/504866887 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40a55f0 con 0x7f8fa8003980 2024-03-16T20:45:29.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fbdffb640 1 -- 172.21.15.122:0/504866887 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb40a5750 con 0x7f8fa8003980 2024-03-16T20:45:29.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fbdffb640 1 -- 172.21.15.122:0/504866887 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40a5ae0 con 0x7f8fa8003980 2024-03-16T20:45:29.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fc5566640 1 -- 172.21.15.122:0/504866887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fa8003980 msgr2=0x7f8fa8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fc5566640 1 --2- 172.21.15.122:0/504866887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fa8003980 0x7f8fa8003da0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009ac0 tx=0x7f8fb401ca20 comp rx=0 tx=0).stop 2024-03-16T20:45:29.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fc5566640 1 -- 172.21.15.122:0/504866887 shutdown_connections 2024-03-16T20:45:29.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fc5566640 1 --2- 172.21.15.122:0/504866887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fa8003980 0x7f8fa8003da0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fc5566640 1 -- 172.21.15.122:0/504866887 >> 172.21.15.122:0/504866887 conn(0x7f8fc00b8860 msgr2=0x7f8fc00b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:29.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fc5566640 1 -- 172.21.15.122:0/504866887 shutdown_connections 2024-03-16T20:45:29.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fc5566640 1 -- 172.21.15.122:0/504866887 wait complete. 2024-03-16T20:45:29.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fc5566640 1 Processor -- start 2024-03-16T20:45:29.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fc5566640 1 -- start start 2024-03-16T20:45:29.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fc5566640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fa8003980 0x7f8fc0146c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fc5566640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fc013e530 0x7f8fc01471b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fc5566640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fc01e1150 0x7f8fc01e1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fc5566640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc013cc00 con 0x7f8fc01e1150 2024-03-16T20:45:29.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fc5566640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc013d080 con 0x7f8fa8003980 2024-03-16T20:45:29.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fc5566640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc013dda0 con 0x7f8fc013e530 2024-03-16T20:45:29.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fbf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fc01e1150 0x7f8fc01e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fbf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fc01e1150 0x7f8fc01e1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34090/0 (socket says 172.21.15.122:34090) 2024-03-16T20:45:29.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fbf7fe640 1 -- 172.21.15.122:0/657189679 learned_addr learned my addr 172.21.15.122:0/657189679 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:29.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fbeffd640 1 --2- 172.21.15.122:0/657189679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fa8003980 0x7f8fc0146c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.964+0000 7f8fbe7fc640 1 --2- 172.21.15.122:0/657189679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fc013e530 0x7f8fc01471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.968+0000 7f8fbf7fe640 1 -- 172.21.15.122:0/657189679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fc013e530 msgr2=0x7f8fc01471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.968+0000 7f8fbf7fe640 1 --2- 172.21.15.122:0/657189679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fc013e530 0x7f8fc01471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.968+0000 7f8fbf7fe640 1 -- 172.21.15.122:0/657189679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fa8003980 msgr2=0x7f8fc0146c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:29.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.968+0000 7f8fbf7fe640 1 --2- 172.21.15.122:0/657189679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fa8003980 0x7f8fc0146c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:29.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.968+0000 7f8fbf7fe640 1 -- 172.21.15.122:0/657189679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc01476f0 con 0x7f8fc01e1150 2024-03-16T20:45:29.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.968+0000 7f8fbf7fe640 1 --2- 172.21.15.122:0/657189679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fc01e1150 0x7f8fc01e1600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00b04d0 tx=0x7f8fb800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:29.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.968+0000 7f8fa77fe640 1 -- 172.21.15.122:0/657189679 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb800eed0 con 0x7f8fc01e1150 2024-03-16T20:45:29.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.968+0000 7f8fa77fe640 1 -- 172.21.15.122:0/657189679 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb8003980 con 0x7f8fc01e1150 2024-03-16T20:45:29.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.968+0000 7f8fa77fe640 1 -- 172.21.15.122:0/657189679 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb8003d50 con 0x7f8fc01e1150 2024-03-16T20:45:29.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.968+0000 7f8fc5566640 1 -- 172.21.15.122:0/657189679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc0148230 con 0x7f8fc01e1150 2024-03-16T20:45:29.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.968+0000 7f8fc5566640 1 -- 172.21.15.122:0/657189679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc0138a70 con 0x7f8fc01e1150 2024-03-16T20:45:29.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.968+0000 7f8fc5566640 1 -- 172.21.15.122:0/657189679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc013cdb0 con 0x7f8fc01e1150 2024-03-16T20:45:29.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.968+0000 7f8fa77fe640 1 -- 172.21.15.122:0/657189679 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f8fb80050d0 con 0x7f8fc01e1150 2024-03-16T20:45:29.973 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.968+0000 7f8fa77fe640 1 --2- 172.21.15.122:0/657189679 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8fac066e70 0x7f8fac069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:29.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.968+0000 7f8fa77fe640 1 -- 172.21.15.122:0/657189679 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fb80845a0 con 0x7f8fc01e1150 2024-03-16T20:45:29.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.968+0000 7f8fbeffd640 1 --2- 172.21.15.122:0/657189679 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8fac066e70 0x7f8fac069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:29.974 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.968+0000 7f8fbeffd640 1 --2- 172.21.15.122:0/657189679 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8fac066e70 0x7f8fac069330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40af000 tx=0x7f8fb40046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:29.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:29.972+0000 7f8fa77fe640 1 -- 172.21.15.122:0/657189679 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8fb8051430 con 0x7f8fc01e1150 2024-03-16T20:45:30.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.132+0000 7f8fc5566640 1 -- 172.21.15.122:0/657189679 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8fc013e250 con 0x7f8fc01e1150 2024-03-16T20:45:30.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.136+0000 7f8fa77fe640 1 -- 172.21.15.122:0/657189679 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f8fc013e250 con 0x7f8fc01e1150 2024-03-16T20:45:30.138 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:30.138 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:06.404479+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:30.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.136+0000 7f8fc5566640 1 -- 172.21.15.122:0/657189679 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8fac066e70 msgr2=0x7f8fac069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.136+0000 7f8fc5566640 1 --2- 172.21.15.122:0/657189679 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8fac066e70 0x7f8fac069330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40af000 tx=0x7f8fb40046d0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.136+0000 7f8fc5566640 1 -- 172.21.15.122:0/657189679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fc01e1150 msgr2=0x7f8fc01e1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.136+0000 7f8fc5566640 1 --2- 172.21.15.122:0/657189679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fc01e1150 0x7f8fc01e1600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00b04d0 tx=0x7f8fb800eb90 comp rx=0 tx=0).stop 2024-03-16T20:45:30.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.136+0000 7f8fc5566640 1 -- 172.21.15.122:0/657189679 shutdown_connections 2024-03-16T20:45:30.140 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.136+0000 7f8fc5566640 1 --2- 172.21.15.122:0/657189679 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8fac066e70 0x7f8fac069330 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.136+0000 7f8fc5566640 1 --2- 172.21.15.122:0/657189679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8fc01e1150 0x7f8fc01e1600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.136+0000 7f8fc5566640 1 --2- 172.21.15.122:0/657189679 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8fc013e530 0x7f8fc01471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.136+0000 7f8fc5566640 1 --2- 172.21.15.122:0/657189679 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8fa8003980 0x7f8fc0146c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.136+0000 7f8fc5566640 1 -- 172.21.15.122:0/657189679 >> 172.21.15.122:0/657189679 conn(0x7f8fc00b8860 msgr2=0x7f8fc00b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:30.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.136+0000 7f8fc5566640 1 -- 172.21.15.122:0/657189679 shutdown_connections 2024-03-16T20:45:30.141 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.136+0000 7f8fc5566640 1 -- 172.21.15.122:0/657189679 wait complete. 2024-03-16T20:45:30.157 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T20:44:52.653523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T20:45:02.758515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T20:45:03.710783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T20:45:30.157 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T20:45:30.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.276+0000 7f8e131ea640 1 Processor -- start 2024-03-16T20:45:30.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.276+0000 7f8e131ea640 1 -- start start 2024-03-16T20:45:30.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.276+0000 7f8e131ea640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e0c141580 0x7f8e0c0baac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.276+0000 7f8e131ea640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8e0c06d350 0x7f8e0c151920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.276+0000 7f8e131ea640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e0c147ff0 con 0x7f8e0c0bb000 2024-03-16T20:45:30.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.276+0000 7f8e131ea640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e0c0bb460 con 0x7f8e0c06d350 2024-03-16T20:45:30.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.276+0000 7f8e131ea640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c1479a0 con 0x7f8e0c141580 2024-03-16T20:45:30.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.276+0000 7f8e10f86640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e0c141580 0x7f8e0c0baac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.276+0000 7f8e10f86640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e0c141580 0x7f8e0c0baac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:50174/0 (socket says 172.21.15.122:50174) 2024-03-16T20:45:30.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.276+0000 7f8e10f86640 1 -- 172.21.15.122:0/1663733219 learned_addr learned my addr 172.21.15.122:0/1663733219 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:30.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.276+0000 7f8e11787640 1 --2- 172.21.15.122:0/1663733219 >> v2:172.21.15.195:3300/0 conn(0x7f8e0c06d350 0x7f8e0c151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.276+0000 7f8e10f86640 1 -- 172.21.15.122:0/1663733219 >> v2:172.21.15.195:3300/0 conn(0x7f8e0c06d350 msgr2=0x7f8e0c151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:30.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.276+0000 7f8e10f86640 1 --2- 172.21.15.122:0/1663733219 >> v2:172.21.15.195:3300/0 conn(0x7f8e0c06d350 0x7f8e0c151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.276+0000 7f8e10f86640 1 -- 172.21.15.122:0/1663733219 >> v1:172.21.15.195:6789/0 conn(0x7f8e0c0bb000 legacy=0x7f8e0c06cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.281 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.276+0000 7f8e10f86640 1 -- 172.21.15.122:0/1663733219 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c147c80 con 0x7f8e0c141580 2024-03-16T20:45:30.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.280+0000 7f8e10f86640 1 --2- 172.21.15.122:0/1663733219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e0c141580 0x7f8e0c0baac0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009870 tx=0x7f8dfc01c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d4e72ab3d5b5833 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:30.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.280+0000 7f8e0b7fe640 1 -- 172.21.15.122:0/1663733219 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0ab670 con 0x7f8e0c141580 2024-03-16T20:45:30.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.280+0000 7f8e0b7fe640 1 -- 172.21.15.122:0/1663733219 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dfc0ab7d0 con 0x7f8e0c141580 2024-03-16T20:45:30.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.280+0000 7f8e0b7fe640 1 -- 172.21.15.122:0/1663733219 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc0abb60 con 0x7f8e0c141580 2024-03-16T20:45:30.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.280+0000 7f8e131ea640 1 -- 172.21.15.122:0/1663733219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e0c141580 msgr2=0x7f8e0c0baac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.280+0000 7f8e131ea640 1 --2- 172.21.15.122:0/1663733219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e0c141580 0x7f8e0c0baac0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc009870 tx=0x7f8dfc01c560 comp rx=0 tx=0).stop 2024-03-16T20:45:30.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.280+0000 7f8e131ea640 1 -- 172.21.15.122:0/1663733219 shutdown_connections 2024-03-16T20:45:30.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.280+0000 7f8e131ea640 1 --2- 172.21.15.122:0/1663733219 >> v2:172.21.15.195:3300/0 conn(0x7f8e0c06d350 0x7f8e0c151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.280+0000 7f8e131ea640 1 --2- 172.21.15.122:0/1663733219 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e0c141580 0x7f8e0c0baac0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.280+0000 7f8e131ea640 1 -- 172.21.15.122:0/1663733219 >> 172.21.15.122:0/1663733219 conn(0x7f8e0c0bca20 msgr2=0x7f8e0c0bce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:30.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.280+0000 7f8e131ea640 1 -- 172.21.15.122:0/1663733219 shutdown_connections 2024-03-16T20:45:30.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.280+0000 7f8e131ea640 1 -- 172.21.15.122:0/1663733219 wait complete. 2024-03-16T20:45:30.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.280+0000 7f8e131ea640 1 Processor -- start 2024-03-16T20:45:30.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.280+0000 7f8e131ea640 1 -- start start 2024-03-16T20:45:30.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e131ea640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e0c06d350 0x7f8e0c0b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e131ea640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c0bb000 0x7f8e0c13c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e131ea640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e0c141580 0x7f8e0c13cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e131ea640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c0b5ce0 con 0x7f8e0c0bb000 2024-03-16T20:45:30.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e131ea640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c1e3ae0 con 0x7f8e0c141580 2024-03-16T20:45:30.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e131ea640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c1e3060 con 0x7f8e0c06d350 2024-03-16T20:45:30.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e0bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c0bb000 0x7f8e0c13c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e0bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c0bb000 0x7f8e0c13c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34104/0 (socket says 172.21.15.122:34104) 2024-03-16T20:45:30.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e0bfff640 1 -- 172.21.15.122:0/978056062 learned_addr learned my addr 172.21.15.122:0/978056062 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:30.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e10f86640 1 --2- 172.21.15.122:0/978056062 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e0c06d350 0x7f8e0c0b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e11787640 1 --2- 172.21.15.122:0/978056062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e0c141580 0x7f8e0c13cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e0bfff640 1 -- 172.21.15.122:0/978056062 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e0c06d350 msgr2=0x7f8e0c0b5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e0bfff640 1 --2- 172.21.15.122:0/978056062 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e0c06d350 0x7f8e0c0b5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e0bfff640 1 -- 172.21.15.122:0/978056062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e0c141580 msgr2=0x7f8e0c13cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e0bfff640 1 --2- 172.21.15.122:0/978056062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e0c141580 0x7f8e0c13cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e0bfff640 1 -- 172.21.15.122:0/978056062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dfc0094d0 con 0x7f8e0c0bb000 2024-03-16T20:45:30.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e10f86640 1 --2- 172.21.15.122:0/978056062 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e0c06d350 0x7f8e0c0b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:30.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e0bfff640 1 --2- 172.21.15.122:0/978056062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c0bb000 0x7f8e0c13c040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000c530 tx=0x7f8e0000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:30.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e097fa640 1 -- 172.21.15.122:0/978056062 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e00005220 con 0x7f8e0c0bb000 2024-03-16T20:45:30.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e097fa640 1 -- 172.21.15.122:0/978056062 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e00005380 con 0x7f8e0c0bb000 2024-03-16T20:45:30.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e097fa640 1 -- 172.21.15.122:0/978056062 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e00007270 con 0x7f8e0c0bb000 2024-03-16T20:45:30.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e131ea640 1 -- 172.21.15.122:0/978056062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0c1e32e0 con 0x7f8e0c0bb000 2024-03-16T20:45:30.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e131ea640 1 -- 172.21.15.122:0/978056062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e0c0b69f0 con 0x7f8e0c0bb000 2024-03-16T20:45:30.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e131ea640 1 -- 172.21.15.122:0/978056062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e0c074bb0 con 0x7f8e0c0bb000 2024-03-16T20:45:30.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e097fa640 1 -- 172.21.15.122:0/978056062 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f8e000073d0 con 0x7f8e0c0bb000 2024-03-16T20:45:30.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e097fa640 1 --2- 172.21.15.122:0/978056062 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8df8066e40 0x7f8df8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e097fa640 1 -- 172.21.15.122:0/978056062 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e000842b0 con 0x7f8e0c0bb000 2024-03-16T20:45:30.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e10f86640 1 --2- 172.21.15.122:0/978056062 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8df8066e40 0x7f8df8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.284+0000 7f8e10f86640 1 --2- 172.21.15.122:0/978056062 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8df8066e40 0x7f8df8069300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0af000 tx=0x7f8dfc0a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:30.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.288+0000 7f8e097fa640 1 -- 172.21.15.122:0/978056062 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8e00051130 con 0x7f8e0c0bb000 2024-03-16T20:45:30.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.436+0000 7f8e131ea640 1 -- 172.21.15.122:0/978056062 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8e0c0b6e50 con 0x7f8e0c0bb000 2024-03-16T20:45:30.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.436+0000 7f8e097fa640 1 -- 172.21.15.122:0/978056062 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e000547f0 con 0x7f8e0c0bb000 2024-03-16T20:45:30.440 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-03-16T20:45:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.440+0000 7f8e131ea640 1 -- 172.21.15.122:0/978056062 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8df8066e40 msgr2=0x7f8df8069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.440+0000 7f8e131ea640 1 --2- 172.21.15.122:0/978056062 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8df8066e40 0x7f8df8069300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc0af000 tx=0x7f8dfc0a7000 comp rx=0 tx=0).stop 2024-03-16T20:45:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.440+0000 7f8e131ea640 1 -- 172.21.15.122:0/978056062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c0bb000 msgr2=0x7f8e0c13c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.440+0000 7f8e131ea640 1 --2- 172.21.15.122:0/978056062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c0bb000 0x7f8e0c13c040 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8e0000c530 tx=0x7f8e0000c9f0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.440+0000 7f8e131ea640 1 -- 172.21.15.122:0/978056062 shutdown_connections 2024-03-16T20:45:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.440+0000 7f8e131ea640 1 --2- 172.21.15.122:0/978056062 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8df8066e40 0x7f8df8069300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.440+0000 7f8e131ea640 1 --2- 172.21.15.122:0/978056062 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8e0c141580 0x7f8e0c13cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.440+0000 7f8e131ea640 1 --2- 172.21.15.122:0/978056062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e0c0bb000 0x7f8e0c13c040 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.440+0000 7f8e131ea640 1 --2- 172.21.15.122:0/978056062 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8e0c06d350 0x7f8e0c0b5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.440+0000 7f8e131ea640 1 -- 172.21.15.122:0/978056062 >> 172.21.15.122:0/978056062 conn(0x7f8e0c0bca20 msgr2=0x7f8e0c0bb660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.440+0000 7f8e131ea640 1 -- 172.21.15.122:0/978056062 shutdown_connections 2024-03-16T20:45:30.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.440+0000 7f8e131ea640 1 -- 172.21.15.122:0/978056062 wait complete. 2024-03-16T20:45:30.458 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T20:45:30.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f8820259640 1 Processor -- start 2024-03-16T20:45:30.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f8820259640 1 -- start start 2024-03-16T20:45:30.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f8820259640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f88180b4020 0x7f88180b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f8820259640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88180b38e0 con 0x7f88180b7820 2024-03-16T20:45:30.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f8820259640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f881809b210 con 0x7f881809ae30 2024-03-16T20:45:30.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f8820259640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88180af790 con 0x7f88180b4020 2024-03-16T20:45:30.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881e7f6640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f88180b7820 0x7f881814e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39810/0 (socket says 172.21.15.122:39810) 2024-03-16T20:45:30.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881e7f6640 1 -- 172.21.15.122:0/2011797210 learned_addr learned my addr 172.21.15.122:0/2011797210 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:30.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881d7f4640 1 --2- 172.21.15.122:0/2011797210 >> v2:172.21.15.195:3300/0 conn(0x7f88180b4020 0x7f88180b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881cff3640 1 -- 172.21.15.122:0/2011797210 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391941716 0 0) 0x7f88180b38e0 con 0x7f88180b7820 2024-03-16T20:45:30.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881cff3640 1 -- 172.21.15.122:0/2011797210 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8804003530 con 0x7f88180b7820 2024-03-16T20:45:30.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881cff3640 1 -- 172.21.15.122:0/2011797210 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1954033368 0 0) 0x7f8804003530 con 0x7f88180b7820 2024-03-16T20:45:30.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881cff3640 1 -- 172.21.15.122:0/2011797210 >> v2:172.21.15.195:3300/0 conn(0x7f88180b4020 msgr2=0x7f88180b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:30.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881cff3640 1 --2- 172.21.15.122:0/2011797210 >> v2:172.21.15.195:3300/0 conn(0x7f88180b4020 0x7f88180b67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881cff3640 1 -- 172.21.15.122:0/2011797210 >> v1:172.21.15.195:6789/0 conn(0x7f881809ae30 legacy=0x7f88180af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881cff3640 1 -- 172.21.15.122:0/2011797210 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88180b4fb0 con 0x7f88180b7820 2024-03-16T20:45:30.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881cff3640 1 -- 172.21.15.122:0/2011797210 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f8810001a00 con 0x7f88180b7820 2024-03-16T20:45:30.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881cff3640 1 -- 172.21.15.122:0/2011797210 >> v1:172.21.15.122:6789/0 conn(0x7f88180b7820 legacy=0x7f881814e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881cff3640 1 --2- 172.21.15.122:0/2011797210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8804003980 0x7f8804003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881cff3640 1 --2- 172.21.15.122:0/2011797210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88040047d0 0x7f8804004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881cff3640 1 --2- 172.21.15.122:0/2011797210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8804005190 0x7f880400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881cff3640 1 -- 172.21.15.122:0/2011797210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88180af790 con 0x7f8804003980 2024-03-16T20:45:30.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881cff3640 1 -- 172.21.15.122:0/2011797210 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880400c130 con 0x7f88040047d0 2024-03-16T20:45:30.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881cff3640 1 -- 172.21.15.122:0/2011797210 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880400c2f0 con 0x7f8804005190 2024-03-16T20:45:30.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881dff5640 1 --2- 172.21.15.122:0/2011797210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8804003980 0x7f8804003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881d7f4640 1 --2- 172.21.15.122:0/2011797210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88040047d0 0x7f8804004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881dff5640 1 -- 172.21.15.122:0/2011797210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8804005190 msgr2=0x7f880400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:30.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881dff5640 1 --2- 172.21.15.122:0/2011797210 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8804005190 0x7f880400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881dff5640 1 -- 172.21.15.122:0/2011797210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88040047d0 msgr2=0x7f8804004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:30.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881dff5640 1 --2- 172.21.15.122:0/2011797210 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f88040047d0 0x7f8804004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881dff5640 1 -- 172.21.15.122:0/2011797210 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8804003530 con 0x7f8804003980 2024-03-16T20:45:30.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.571+0000 7f881e7f6640 1 -- 172.21.15.122:0/2011797210 reap_dead start 2024-03-16T20:45:30.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f881dff5640 1 --2- 172.21.15.122:0/2011797210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8804003980 0x7f8804003da0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8814001780 tx=0x7f881401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eab36891a67985d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:30.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f881cff3640 1 -- 172.21.15.122:0/2011797210 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88140052d0 con 0x7f8804003980 2024-03-16T20:45:30.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f881cff3640 1 -- 172.21.15.122:0/2011797210 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88140a7050 con 0x7f8804003980 2024-03-16T20:45:30.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f881cff3640 1 -- 172.21.15.122:0/2011797210 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8814003600 con 0x7f8804003980 2024-03-16T20:45:30.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f8820259640 1 -- 172.21.15.122:0/2011797210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8804003980 msgr2=0x7f8804003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f8820259640 1 --2- 172.21.15.122:0/2011797210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8804003980 0x7f8804003da0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8814001780 tx=0x7f881401c3d0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.580 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f8820259640 1 -- 172.21.15.122:0/2011797210 shutdown_connections 2024-03-16T20:45:30.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f8820259640 1 --2- 172.21.15.122:0/2011797210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8804003980 0x7f8804003da0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f8820259640 1 -- 172.21.15.122:0/2011797210 >> 172.21.15.122:0/2011797210 conn(0x7f88180b8d10 msgr2=0x7f881809a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:30.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f8820259640 1 -- 172.21.15.122:0/2011797210 shutdown_connections 2024-03-16T20:45:30.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f8820259640 1 -- 172.21.15.122:0/2011797210 wait complete. 2024-03-16T20:45:30.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f8820259640 1 Processor -- start 2024-03-16T20:45:30.581 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f8820259640 1 -- start start 2024-03-16T20:45:30.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f8820259640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8804003980 0x7f881809cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f8820259640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88180b20f0 0x7f88180b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f8820259640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88180b18f0 0x7f8818144350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f8820259640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f881806fda0 con 0x7f88180b18f0 2024-03-16T20:45:30.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f8820259640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88180b1d70 con 0x7f8804003980 2024-03-16T20:45:30.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f8820259640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8818072c90 con 0x7f88180b20f0 2024-03-16T20:45:30.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f881e7f6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88180b18f0 0x7f8818144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f881e7f6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88180b18f0 0x7f8818144350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34128/0 (socket says 172.21.15.122:34128) 2024-03-16T20:45:30.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f881e7f6640 1 -- 172.21.15.122:0/1115589819 learned_addr learned my addr 172.21.15.122:0/1115589819 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:30.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f881dff5640 1 --2- 172.21.15.122:0/1115589819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8804003980 0x7f881809cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f881d7f4640 1 --2- 172.21.15.122:0/1115589819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88180b20f0 0x7f88180b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f881e7f6640 1 -- 172.21.15.122:0/1115589819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88180b20f0 msgr2=0x7f88180b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f881e7f6640 1 --2- 172.21.15.122:0/1115589819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88180b20f0 0x7f88180b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f881e7f6640 1 -- 172.21.15.122:0/1115589819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8804003980 msgr2=0x7f881809cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f881e7f6640 1 --2- 172.21.15.122:0/1115589819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8804003980 0x7f881809cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f881e7f6640 1 -- 172.21.15.122:0/1115589819 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8818070dd0 con 0x7f88180b18f0 2024-03-16T20:45:30.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f881dff5640 1 --2- 172.21.15.122:0/1115589819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8804003980 0x7f881809cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:30.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f881e7f6640 1 --2- 172.21.15.122:0/1115589819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88180b18f0 0x7f8818144350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8810001570 tx=0x7f881000ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:30.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f880e7fc640 1 -- 172.21.15.122:0/1115589819 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881000a610 con 0x7f88180b18f0 2024-03-16T20:45:30.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f880e7fc640 1 -- 172.21.15.122:0/1115589819 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881000a770 con 0x7f88180b18f0 2024-03-16T20:45:30.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f8820259640 1 -- 172.21.15.122:0/1115589819 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881809d4d0 con 0x7f88180b18f0 2024-03-16T20:45:30.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f880e7fc640 1 -- 172.21.15.122:0/1115589819 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881000ab20 con 0x7f88180b18f0 2024-03-16T20:45:30.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.575+0000 7f8820259640 1 -- 172.21.15.122:0/1115589819 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8818144890 con 0x7f88180b18f0 2024-03-16T20:45:30.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.579+0000 7f8820259640 1 -- 172.21.15.122:0/1115589819 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88180b2f40 con 0x7f88180b18f0 2024-03-16T20:45:30.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.579+0000 7f880e7fc640 1 -- 172.21.15.122:0/1115589819 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f881000ac80 con 0x7f88180b18f0 2024-03-16T20:45:30.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.579+0000 7f880e7fc640 1 --2- 172.21.15.122:0/1115589819 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f87fc066e40 0x7f87fc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.579+0000 7f880e7fc640 1 -- 172.21.15.122:0/1115589819 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88100846d0 con 0x7f88180b18f0 2024-03-16T20:45:30.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.579+0000 7f881dff5640 1 --2- 172.21.15.122:0/1115589819 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f87fc066e40 0x7f87fc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.579+0000 7f881dff5640 1 --2- 172.21.15.122:0/1115589819 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f87fc066e40 0x7f87fc069300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8814001750 tx=0x7f88140011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:30.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.583+0000 7f880e7fc640 1 -- 172.21.15.122:0/1115589819 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f88100514b0 con 0x7f88180b18f0 2024-03-16T20:45:30.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.731+0000 7f8820259640 1 -- 172.21.15.122:0/1115589819 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f881806fb00 con 0x7f88180b18f0 2024-03-16T20:45:30.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.731+0000 7f880e7fc640 1 -- 172.21.15.122:0/1115589819 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8810054b70 con 0x7f88180b18f0 2024-03-16T20:45:30.736 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-03-16T20:45:30.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.735+0000 7f8820259640 1 -- 172.21.15.122:0/1115589819 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f87fc066e40 msgr2=0x7f87fc069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.735+0000 7f8820259640 1 --2- 172.21.15.122:0/1115589819 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f87fc066e40 0x7f87fc069300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8814001750 tx=0x7f88140011a0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.735+0000 7f8820259640 1 -- 172.21.15.122:0/1115589819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88180b18f0 msgr2=0x7f8818144350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.735+0000 7f8820259640 1 --2- 172.21.15.122:0/1115589819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88180b18f0 0x7f8818144350 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8810001570 tx=0x7f881000ce80 comp rx=0 tx=0).stop 2024-03-16T20:45:30.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.735+0000 7f8820259640 1 -- 172.21.15.122:0/1115589819 shutdown_connections 2024-03-16T20:45:30.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.735+0000 7f8820259640 1 --2- 172.21.15.122:0/1115589819 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f87fc066e40 0x7f87fc069300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.735+0000 7f8820259640 1 --2- 172.21.15.122:0/1115589819 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88180b18f0 0x7f8818144350 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.735+0000 7f8820259640 1 --2- 172.21.15.122:0/1115589819 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f88180b20f0 0x7f88180b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.735+0000 7f8820259640 1 --2- 172.21.15.122:0/1115589819 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8804003980 0x7f881809cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.735+0000 7f8820259640 1 -- 172.21.15.122:0/1115589819 >> 172.21.15.122:0/1115589819 conn(0x7f88180b8d10 msgr2=0x7f881806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:30.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.735+0000 7f8820259640 1 -- 172.21.15.122:0/1115589819 shutdown_connections 2024-03-16T20:45:30.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.735+0000 7f8820259640 1 -- 172.21.15.122:0/1115589819 wait complete. 2024-03-16T20:45:30.753 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T20:45:30.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb161a92640 1 Processor -- start 2024-03-16T20:45:30.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb161a92640 1 -- start start 2024-03-16T20:45:30.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb161a92640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0b7540 0x7fb15c147110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb161a92640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb15c1418e0 con 0x7fb15c06cde0 2024-03-16T20:45:30.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb161a92640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb15c06c130 con 0x7fb15c147d60 2024-03-16T20:45:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb161a92640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0b8270 con 0x7fb15c0b7540 2024-03-16T20:45:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15affd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fb15c06cde0 0x7fb15c147650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39816/0 (socket says 172.21.15.122:39816) 2024-03-16T20:45:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15affd640 1 -- 172.21.15.122:0/256768467 learned_addr learned my addr 172.21.15.122:0/256768467 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15b7fe640 1 --2- 172.21.15.122:0/256768467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0b7540 0x7fb15c147110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15a7fc640 1 -- 172.21.15.122:0/256768467 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099618860 0 0) 0x7fb14c001120 con 0x7fb15c06cde0 2024-03-16T20:45:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15a7fc640 1 -- 172.21.15.122:0/256768467 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb144003530 con 0x7fb15c06cde0 2024-03-16T20:45:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15a7fc640 1 -- 172.21.15.122:0/256768467 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1407072273 0 0) 0x7fb144003530 con 0x7fb15c06cde0 2024-03-16T20:45:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15a7fc640 1 -- 172.21.15.122:0/256768467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0b7540 msgr2=0x7fb15c147110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15a7fc640 1 --2- 172.21.15.122:0/256768467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0b7540 0x7fb15c147110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15a7fc640 1 -- 172.21.15.122:0/256768467 >> v1:172.21.15.195:6789/0 conn(0x7fb15c147d60 legacy=0x7fb15c151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15a7fc640 1 -- 172.21.15.122:0/256768467 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c0b7920 con 0x7fb15c06cde0 2024-03-16T20:45:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15a7fc640 1 -- 172.21.15.122:0/256768467 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7fb14c001810 con 0x7fb15c06cde0 2024-03-16T20:45:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15a7fc640 1 -- 172.21.15.122:0/256768467 >> v1:172.21.15.122:6789/0 conn(0x7fb15c06cde0 legacy=0x7fb15c147650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.870 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15a7fc640 1 --2- 172.21.15.122:0/256768467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0b7540 0x7fb1440039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15a7fc640 1 --2- 172.21.15.122:0/256768467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb144003f80 0x7fb1440047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15a7fc640 1 --2- 172.21.15.122:0/256768467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb144004d10 0x7fb144005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15a7fc640 1 -- 172.21.15.122:0/256768467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0b8270 con 0x7fb144004d10 2024-03-16T20:45:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15a7fc640 1 -- 172.21.15.122:0/256768467 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14400bad0 con 0x7fb144003f80 2024-03-16T20:45:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15a7fc640 1 -- 172.21.15.122:0/256768467 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14400bc90 con 0x7fb15c0b7540 2024-03-16T20:45:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15bfff640 1 --2- 172.21.15.122:0/256768467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb144004d10 0x7fb144005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15affd640 1 --2- 172.21.15.122:0/256768467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb144003f80 0x7fb1440047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15b7fe640 1 --2- 172.21.15.122:0/256768467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0b7540 0x7fb1440039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15bfff640 1 -- 172.21.15.122:0/256768467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0b7540 msgr2=0x7fb1440039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15bfff640 1 --2- 172.21.15.122:0/256768467 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb15c0b7540 0x7fb1440039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15bfff640 1 -- 172.21.15.122:0/256768467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb144003f80 msgr2=0x7fb1440047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15bfff640 1 --2- 172.21.15.122:0/256768467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb144003f80 0x7fb1440047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15bfff640 1 -- 172.21.15.122:0/256768467 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb144003530 con 0x7fb144004d10 2024-03-16T20:45:30.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.863+0000 7fb15affd640 1 --2- 172.21.15.122:0/256768467 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb144003f80 0x7fb1440047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:30.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.867+0000 7fb15bfff640 1 --2- 172.21.15.122:0/256768467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb144004d10 0x7fb144005190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0b6de0 tx=0x7fb15401c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9dbb27985d985a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:30.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.867+0000 7fb15a7fc640 1 -- 172.21.15.122:0/256768467 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb154003b80 con 0x7fb144004d10 2024-03-16T20:45:30.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.867+0000 7fb15a7fc640 1 -- 172.21.15.122:0/256768467 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb154003ce0 con 0x7fb144004d10 2024-03-16T20:45:30.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.867+0000 7fb15a7fc640 1 -- 172.21.15.122:0/256768467 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1540052c0 con 0x7fb144004d10 2024-03-16T20:45:30.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.867+0000 7fb161a92640 1 -- 172.21.15.122:0/256768467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb144004d10 msgr2=0x7fb144005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.867+0000 7fb161a92640 1 --2- 172.21.15.122:0/256768467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb144004d10 0x7fb144005190 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fb15c0b6de0 tx=0x7fb15401c720 comp rx=0 tx=0).stop 2024-03-16T20:45:30.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.867+0000 7fb15bfff640 1 -- 172.21.15.122:0/256768467 reap_dead start 2024-03-16T20:45:30.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.867+0000 7fb161a92640 1 -- 172.21.15.122:0/256768467 shutdown_connections 2024-03-16T20:45:30.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.867+0000 7fb161a92640 1 -- 172.21.15.122:0/256768467 >> 172.21.15.122:0/256768467 conn(0x7fb15c06fc00 msgr2=0x7fb15c070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:30.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.867+0000 7fb161a92640 1 -- 172.21.15.122:0/256768467 shutdown_connections 2024-03-16T20:45:30.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.867+0000 7fb161a92640 1 -- 172.21.15.122:0/256768467 wait complete. 2024-03-16T20:45:30.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.867+0000 7fb161a92640 1 Processor -- start 2024-03-16T20:45:30.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.867+0000 7fb161a92640 1 -- start start 2024-03-16T20:45:30.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb161a92640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb144004d10 0x7fb15c144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb161a92640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb15c144b60 0x7fb15c1dc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb161a92640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb15c0b0ee0 0x7fb15c0b1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb161a92640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c1dcad0 con 0x7fb15c144b60 2024-03-16T20:45:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb161a92640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0ac8d0 con 0x7fb15c0b0ee0 2024-03-16T20:45:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb161a92640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0acb20 con 0x7fb144004d10 2024-03-16T20:45:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb15affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb15c144b60 0x7fb15c1dc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb15affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb15c144b60 0x7fb15c1dc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34158/0 (socket says 172.21.15.122:34158) 2024-03-16T20:45:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb15affd640 1 -- 172.21.15.122:0/3658083406 learned_addr learned my addr 172.21.15.122:0/3658083406 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb15bfff640 1 --2- 172.21.15.122:0/3658083406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb15c0b0ee0 0x7fb15c0b1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb15b7fe640 1 --2- 172.21.15.122:0/3658083406 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb144004d10 0x7fb15c144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb15affd640 1 -- 172.21.15.122:0/3658083406 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb144004d10 msgr2=0x7fb15c144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb15affd640 1 --2- 172.21.15.122:0/3658083406 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb144004d10 0x7fb15c144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb15affd640 1 -- 172.21.15.122:0/3658083406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb15c0b0ee0 msgr2=0x7fb15c0b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb15affd640 1 --2- 172.21.15.122:0/3658083406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb15c0b0ee0 0x7fb15c0b1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:30.876 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb15affd640 1 -- 172.21.15.122:0/3658083406 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c0ae910 con 0x7fb15c144b60 2024-03-16T20:45:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb15affd640 1 --2- 172.21.15.122:0/3658083406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb15c144b60 0x7fb15c1dc590 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00e7b0 tx=0x7fb14c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb143fff640 1 -- 172.21.15.122:0/3658083406 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c00eec0 con 0x7fb15c144b60 2024-03-16T20:45:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb143fff640 1 -- 172.21.15.122:0/3658083406 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14c00c870 con 0x7fb15c144b60 2024-03-16T20:45:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb161a92640 1 -- 172.21.15.122:0/3658083406 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15c0ad3f0 con 0x7fb15c144b60 2024-03-16T20:45:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb143fff640 1 -- 172.21.15.122:0/3658083406 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c00cbe0 con 0x7fb15c144b60 2024-03-16T20:45:30.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb161a92640 1 -- 172.21.15.122:0/3658083406 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15c0ae630 con 0x7fb15c144b60 2024-03-16T20:45:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb161a92640 1 -- 172.21.15.122:0/3658083406 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15c0ad910 con 0x7fb15c144b60 2024-03-16T20:45:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb143fff640 1 -- 172.21.15.122:0/3658083406 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fb14c00cd40 con 0x7fb15c144b60 2024-03-16T20:45:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb143fff640 1 --2- 172.21.15.122:0/3658083406 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb148066e40 0x7fb148069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb143fff640 1 -- 172.21.15.122:0/3658083406 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb14c084800 con 0x7fb15c144b60 2024-03-16T20:45:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.871+0000 7fb15b7fe640 1 --2- 172.21.15.122:0/3658083406 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb148066e40 0x7fb148069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:30.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.875+0000 7fb15b7fe640 1 --2- 172.21.15.122:0/3658083406 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb148066e40 0x7fb148069300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb150009a30 tx=0x7fb150008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:30.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:30.875+0000 7fb143fff640 1 -- 172.21.15.122:0/3658083406 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb15c0ad910 con 0x7fb15c144b60 2024-03-16T20:45:31.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.031+0000 7fb161a92640 1 -- 172.21.15.122:0/3658083406 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb15c1418e0 con 0x7fb15c144b60 2024-03-16T20:45:31.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.035+0000 7fb143fff640 1 -- 172.21.15.122:0/3658083406 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb15c1418e0 con 0x7fb15c144b60 2024-03-16T20:45:31.038 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-03-16T20:45:31.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.035+0000 7fb161a92640 1 -- 172.21.15.122:0/3658083406 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb148066e40 msgr2=0x7fb148069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:31.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.035+0000 7fb161a92640 1 --2- 172.21.15.122:0/3658083406 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb148066e40 0x7fb148069300 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb150009a30 tx=0x7fb150008040 comp rx=0 tx=0).stop 2024-03-16T20:45:31.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.035+0000 7fb161a92640 1 -- 172.21.15.122:0/3658083406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb15c144b60 msgr2=0x7fb15c1dc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:31.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.035+0000 7fb161a92640 1 --2- 172.21.15.122:0/3658083406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb15c144b60 0x7fb15c1dc590 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb14c00e7b0 tx=0x7fb14c00ec70 comp rx=0 tx=0).stop 2024-03-16T20:45:31.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.035+0000 7fb161a92640 1 -- 172.21.15.122:0/3658083406 shutdown_connections 2024-03-16T20:45:31.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.035+0000 7fb161a92640 1 --2- 172.21.15.122:0/3658083406 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb148066e40 0x7fb148069300 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.035+0000 7fb161a92640 1 --2- 172.21.15.122:0/3658083406 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb15c0b0ee0 0x7fb15c0b1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.035+0000 7fb161a92640 1 --2- 172.21.15.122:0/3658083406 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb15c144b60 0x7fb15c1dc590 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.035+0000 7fb161a92640 1 --2- 172.21.15.122:0/3658083406 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb144004d10 0x7fb15c144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.035+0000 7fb161a92640 1 -- 172.21.15.122:0/3658083406 >> 172.21.15.122:0/3658083406 conn(0x7fb15c06fc00 msgr2=0x7fb15c146e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:31.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.035+0000 7fb161a92640 1 -- 172.21.15.122:0/3658083406 shutdown_connections 2024-03-16T20:45:31.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.035+0000 7fb161a92640 1 -- 172.21.15.122:0/3658083406 wait complete. 2024-03-16T20:45:31.055 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T20:45:31.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.159+0000 7f68cc01d640 1 Processor -- start 2024-03-16T20:45:31.163 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.159+0000 7f68cc01d640 1 -- start start 2024-03-16T20:45:31.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.159+0000 7f68cc01d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f68c40b82b0 0x7f68c40b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.159+0000 7f68cc01d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f68c41422e0 0x7f68c4142770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.159+0000 7f68cc01d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c40b3fa0 con 0x7f68c409ae30 2024-03-16T20:45:31.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.159+0000 7f68cc01d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c40b50a0 con 0x7f68c40b82b0 2024-03-16T20:45:31.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.159+0000 7f68cc01d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c4142cb0 con 0x7f68c41422e0 2024-03-16T20:45:31.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.159+0000 7f68c95b8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f68c40b82b0 0x7f68c40b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.159+0000 7f68c95b8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f68c40b82b0 0x7f68c40b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34164/0 (socket says 172.21.15.122:34164) 2024-03-16T20:45:31.164 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.159+0000 7f68c95b8640 1 -- 172.21.15.122:0/631176413 learned_addr learned my addr 172.21.15.122:0/631176413 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:31.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.159+0000 7f68c95b8640 1 -- 172.21.15.122:0/631176413 >> v2:172.21.15.195:3300/0 conn(0x7f68c41422e0 msgr2=0x7f68c4142770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:31.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.159+0000 7f68ca5ba640 1 --2- 172.21.15.122:0/631176413 >> v2:172.21.15.195:3300/0 conn(0x7f68c41422e0 0x7f68c4142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.159+0000 7f68c95b8640 1 --2- 172.21.15.122:0/631176413 >> v2:172.21.15.195:3300/0 conn(0x7f68c41422e0 0x7f68c4142770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.159+0000 7f68c95b8640 1 -- 172.21.15.122:0/631176413 >> v1:172.21.15.195:6789/0 conn(0x7f68c409ae30 legacy=0x7f68c40b7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:31.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.159+0000 7f68c95b8640 1 -- 172.21.15.122:0/631176413 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c409b210 con 0x7f68c40b82b0 2024-03-16T20:45:31.165 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.159+0000 7f68ca5ba640 1 --2- 172.21.15.122:0/631176413 >> v2:172.21.15.195:3300/0 conn(0x7f68c41422e0 0x7f68c4142770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:31.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68c95b8640 1 --2- 172.21.15.122:0/631176413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68c40b82b0 0x7f68c40b4430 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f68c409b400 tx=0x7f68c001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7e0b1d322115f2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:31.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68c8db7640 1 -- 172.21.15.122:0/631176413 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c00aa640 con 0x7f68c40b82b0 2024-03-16T20:45:31.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68c8db7640 1 -- 172.21.15.122:0/631176413 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c00aa7a0 con 0x7f68c40b82b0 2024-03-16T20:45:31.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68c8db7640 1 -- 172.21.15.122:0/631176413 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c00aab30 con 0x7f68c40b82b0 2024-03-16T20:45:31.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 -- 172.21.15.122:0/631176413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68c40b82b0 msgr2=0x7f68c40b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:31.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 --2- 172.21.15.122:0/631176413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68c40b82b0 0x7f68c40b4430 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f68c409b400 tx=0x7f68c001c4e0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 -- 172.21.15.122:0/631176413 shutdown_connections 2024-03-16T20:45:31.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 --2- 172.21.15.122:0/631176413 >> v2:172.21.15.195:3300/0 conn(0x7f68c41422e0 0x7f68c4142770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 --2- 172.21.15.122:0/631176413 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68c40b82b0 0x7f68c40b4430 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 -- 172.21.15.122:0/631176413 >> 172.21.15.122:0/631176413 conn(0x7f68c40b8d10 msgr2=0x7f68c409a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:31.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 -- 172.21.15.122:0/631176413 shutdown_connections 2024-03-16T20:45:31.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 -- 172.21.15.122:0/631176413 wait complete. 2024-03-16T20:45:31.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 Processor -- start 2024-03-16T20:45:31.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 -- start start 2024-03-16T20:45:31.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68c409ae30 0x7f68c40b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68c40b82b0 0x7f68c41e1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68c41422e0 0x7f68c41e1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c40b1300 con 0x7f68c40b82b0 2024-03-16T20:45:31.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c41e0c70 con 0x7f68c41422e0 2024-03-16T20:45:31.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c41488c0 con 0x7f68c409ae30 2024-03-16T20:45:31.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68c95b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68c40b82b0 0x7f68c41e1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68c95b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68c40b82b0 0x7f68c41e1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34172/0 (socket says 172.21.15.122:34172) 2024-03-16T20:45:31.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68c95b8640 1 -- 172.21.15.122:0/180176405 learned_addr learned my addr 172.21.15.122:0/180176405 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:31.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68c9db9640 1 --2- 172.21.15.122:0/180176405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68c409ae30 0x7f68c40b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68ca5ba640 1 --2- 172.21.15.122:0/180176405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68c41422e0 0x7f68c41e1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68c95b8640 1 -- 172.21.15.122:0/180176405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68c409ae30 msgr2=0x7f68c40b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:31.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68c95b8640 1 --2- 172.21.15.122:0/180176405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68c409ae30 0x7f68c40b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68c95b8640 1 -- 172.21.15.122:0/180176405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68c41422e0 msgr2=0x7f68c41e1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:31.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68c95b8640 1 --2- 172.21.15.122:0/180176405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68c41422e0 0x7f68c41e1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68c95b8640 1 -- 172.21.15.122:0/180176405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c00094d0 con 0x7f68c40b82b0 2024-03-16T20:45:31.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68ca5ba640 1 --2- 172.21.15.122:0/180176405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68c41422e0 0x7f68c41e1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:31.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68c95b8640 1 --2- 172.21.15.122:0/180176405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68c40b82b0 0x7f68c41e1150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f68c001a110 tx=0x7f68c001ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:31.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68ba7fc640 1 -- 172.21.15.122:0/180176405 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c00b36b0 con 0x7f68c40b82b0 2024-03-16T20:45:31.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68ba7fc640 1 -- 172.21.15.122:0/180176405 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c00b3810 con 0x7f68c40b82b0 2024-03-16T20:45:31.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 -- 172.21.15.122:0/180176405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c4072a90 con 0x7f68c40b82b0 2024-03-16T20:45:31.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68ba7fc640 1 -- 172.21.15.122:0/180176405 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c00b3ba0 con 0x7f68c40b82b0 2024-03-16T20:45:31.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.163+0000 7f68cc01d640 1 -- 172.21.15.122:0/180176405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c40b0ae0 con 0x7f68c40b82b0 2024-03-16T20:45:31.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.167+0000 7f68cc01d640 1 -- 172.21.15.122:0/180176405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c4074bb0 con 0x7f68c40b82b0 2024-03-16T20:45:31.172 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.167+0000 7f68ba7fc640 1 -- 172.21.15.122:0/180176405 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f68c00a8050 con 0x7f68c40b82b0 2024-03-16T20:45:31.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.167+0000 7f68ba7fc640 1 --2- 172.21.15.122:0/180176405 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f68a8066e40 0x7f68a8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.167+0000 7f68ba7fc640 1 -- 172.21.15.122:0/180176405 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68c011f130 con 0x7f68c40b82b0 2024-03-16T20:45:31.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.167+0000 7f68c9db9640 1 --2- 172.21.15.122:0/180176405 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f68a8066e40 0x7f68a8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.167+0000 7f68c9db9640 1 --2- 172.21.15.122:0/180176405 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f68a8066e40 0x7f68a8069300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f68b4007fd0 tx=0x7f68b4007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:31.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.171+0000 7f68ba7fc640 1 -- 172.21.15.122:0/180176405 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f68c00ec100 con 0x7f68c40b82b0 2024-03-16T20:45:31.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.327+0000 7f68cc01d640 1 -- 172.21.15.122:0/180176405 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f68c4099110 con 0x7f68c40b82b0 2024-03-16T20:45:31.333 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.327+0000 7f68ba7fc640 1 -- 172.21.15.122:0/180176405 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f68c00ef7c0 con 0x7f68c40b82b0 2024-03-16T20:45:31.333 INFO:teuthology.orchestra.run.smithi122.stderr:listed 0 entries 2024-03-16T20:45:31.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.331+0000 7f68cc01d640 1 -- 172.21.15.122:0/180176405 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f68a8066e40 msgr2=0x7f68a8069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:31.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.331+0000 7f68cc01d640 1 --2- 172.21.15.122:0/180176405 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f68a8066e40 0x7f68a8069300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f68b4007fd0 tx=0x7f68b4007e40 comp rx=0 tx=0).stop 2024-03-16T20:45:31.335 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.331+0000 7f68cc01d640 1 -- 172.21.15.122:0/180176405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68c40b82b0 msgr2=0x7f68c41e1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:31.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.331+0000 7f68cc01d640 1 --2- 172.21.15.122:0/180176405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68c40b82b0 0x7f68c41e1150 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f68c001a110 tx=0x7f68c001ad20 comp rx=0 tx=0).stop 2024-03-16T20:45:31.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.331+0000 7f68cc01d640 1 -- 172.21.15.122:0/180176405 shutdown_connections 2024-03-16T20:45:31.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.331+0000 7f68cc01d640 1 --2- 172.21.15.122:0/180176405 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f68a8066e40 0x7f68a8069300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.331+0000 7f68cc01d640 1 --2- 172.21.15.122:0/180176405 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f68c41422e0 0x7f68c41e1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.331+0000 7f68cc01d640 1 --2- 172.21.15.122:0/180176405 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f68c40b82b0 0x7f68c41e1150 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.331+0000 7f68cc01d640 1 --2- 172.21.15.122:0/180176405 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f68c409ae30 0x7f68c40b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.331+0000 7f68cc01d640 1 -- 172.21.15.122:0/180176405 >> 172.21.15.122:0/180176405 conn(0x7f68c40b8d10 msgr2=0x7f68c4099bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:31.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.331+0000 7f68cc01d640 1 -- 172.21.15.122:0/180176405 shutdown_connections 2024-03-16T20:45:31.336 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.331+0000 7f68cc01d640 1 -- 172.21.15.122:0/180176405 wait complete. 2024-03-16T20:45:31.350 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi122.front.sepia.ceph.com... 2024-03-16T20:45:31.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:31.351 DEBUG:teuthology.orchestra.run.smithi122:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-16T20:45:31.379 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:45:31.380 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-03-16T20:45:31.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:31.380 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-16T20:45:31.432 INFO:tasks.cephfs.cephfs_test_case:[] 2024-03-16T20:45:31.433 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T20:45:31.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.591+0000 7fcadd43b640 1 Processor -- start 2024-03-16T20:45:31.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.591+0000 7fcadd43b640 1 -- start start 2024-03-16T20:45:31.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.591+0000 7fcadd43b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcad81411e0 0x7fcad8141660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.591+0000 7fcadd43b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcad814fc20 0x7fcad8152010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.591+0000 7fcadd43b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcad806f760 con 0x7fcad806b1f0 2024-03-16T20:45:31.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.591+0000 7fcadd43b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcad806f0d0 con 0x7fcad81411e0 2024-03-16T20:45:31.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.591+0000 7fcadd43b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad81482d0 con 0x7fcad814fc20 2024-03-16T20:45:31.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.591+0000 7fcad67fc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fcad81411e0 0x7fcad8141660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.591+0000 7fcad6ffd640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fcad806b1f0 0x7fcad8147bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.122:57678/0 (socket says 172.21.15.122:57678) 2024-03-16T20:45:31.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.591+0000 7fcad6ffd640 1 -- 172.21.15.122:0/3984756312 learned_addr learned my addr 172.21.15.122:0/3984756312 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:31.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.591+0000 7fcad77fe640 1 --2- 172.21.15.122:0/3984756312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcad814fc20 0x7fcad8152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.591+0000 7fcad67fc640 1 -- 172.21.15.122:0/3984756312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcad814fc20 msgr2=0x7fcad8152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:31.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.591+0000 7fcad67fc640 1 --2- 172.21.15.122:0/3984756312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcad814fc20 0x7fcad8152010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.591+0000 7fcad67fc640 1 -- 172.21.15.122:0/3984756312 >> v1:172.21.15.195:6789/0 conn(0x7fcad806b1f0 legacy=0x7fcad8147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:31.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.591+0000 7fcad67fc640 1 -- 172.21.15.122:0/3984756312 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad8147730 con 0x7fcad81411e0 2024-03-16T20:45:31.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcad67fc640 1 --2- 172.21.15.122:0/3984756312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcad81411e0 0x7fcad8141660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0097c0 tx=0x7fcacc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deb2628ca80218ae server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:31.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcad5ffb640 1 -- 172.21.15.122:0/3984756312 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc0ab600 con 0x7fcad81411e0 2024-03-16T20:45:31.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcad5ffb640 1 -- 172.21.15.122:0/3984756312 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc0a7050 con 0x7fcad81411e0 2024-03-16T20:45:31.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcad5ffb640 1 -- 172.21.15.122:0/3984756312 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc0ab980 con 0x7fcad81411e0 2024-03-16T20:45:31.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcadd43b640 1 -- 172.21.15.122:0/3984756312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcad81411e0 msgr2=0x7fcad8141660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:31.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcadd43b640 1 --2- 172.21.15.122:0/3984756312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcad81411e0 0x7fcad8141660 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0097c0 tx=0x7fcacc01c4a0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcadd43b640 1 -- 172.21.15.122:0/3984756312 shutdown_connections 2024-03-16T20:45:31.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcadd43b640 1 --2- 172.21.15.122:0/3984756312 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcad814fc20 0x7fcad8152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcadd43b640 1 --2- 172.21.15.122:0/3984756312 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcad81411e0 0x7fcad8141660 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcadd43b640 1 -- 172.21.15.122:0/3984756312 >> 172.21.15.122:0/3984756312 conn(0x7fcad8070d30 msgr2=0x7fcad8071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:31.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcadd43b640 1 -- 172.21.15.122:0/3984756312 shutdown_connections 2024-03-16T20:45:31.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcadd43b640 1 -- 172.21.15.122:0/3984756312 wait complete. 2024-03-16T20:45:31.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcadd43b640 1 Processor -- start 2024-03-16T20:45:31.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcadd43b640 1 -- start start 2024-03-16T20:45:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcadd43b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcad806b1f0 0x7fcad814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcadd43b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcad81411e0 0x7fcad81dfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcadd43b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcad814fc20 0x7fcad8151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcadd43b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad80b1180 con 0x7fcad81411e0 2024-03-16T20:45:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcadd43b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad814b510 con 0x7fcad814fc20 2024-03-16T20:45:31.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.595+0000 7fcadd43b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcad80b1e70 con 0x7fcad806b1f0 2024-03-16T20:45:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcad67fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcad81411e0 0x7fcad81dfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcad67fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcad81411e0 0x7fcad81dfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34174/0 (socket says 172.21.15.122:34174) 2024-03-16T20:45:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcad67fc640 1 -- 172.21.15.122:0/1520880338 learned_addr learned my addr 172.21.15.122:0/1520880338 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcad6ffd640 1 --2- 172.21.15.122:0/1520880338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcad806b1f0 0x7fcad814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcad77fe640 1 --2- 172.21.15.122:0/1520880338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcad814fc20 0x7fcad8151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcad67fc640 1 -- 172.21.15.122:0/1520880338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcad806b1f0 msgr2=0x7fcad814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcad67fc640 1 --2- 172.21.15.122:0/1520880338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcad806b1f0 0x7fcad814afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcad67fc640 1 -- 172.21.15.122:0/1520880338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcad814fc20 msgr2=0x7fcad8151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcad67fc640 1 --2- 172.21.15.122:0/1520880338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcad814fc20 0x7fcad8151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcad67fc640 1 -- 172.21.15.122:0/1520880338 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc009420 con 0x7fcad81411e0 2024-03-16T20:45:31.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcad67fc640 1 --2- 172.21.15.122:0/1520880338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcad81411e0 0x7fcad81dfbe0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0ae040 tx=0x7fcacc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:31.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcabf7fe640 1 -- 172.21.15.122:0/1520880338 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc0b9070 con 0x7fcad81411e0 2024-03-16T20:45:31.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcabf7fe640 1 -- 172.21.15.122:0/1520880338 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcacc0b4640 con 0x7fcad81411e0 2024-03-16T20:45:31.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcabf7fe640 1 -- 172.21.15.122:0/1520880338 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcacc0b49f0 con 0x7fcad81411e0 2024-03-16T20:45:31.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcadd43b640 1 -- 172.21.15.122:0/1520880338 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad80b76d0 con 0x7fcad81411e0 2024-03-16T20:45:31.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcadd43b640 1 -- 172.21.15.122:0/1520880338 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad81e3590 con 0x7fcad81411e0 2024-03-16T20:45:31.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcadd43b640 1 -- 172.21.15.122:0/1520880338 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad80b7530 con 0x7fcad81411e0 2024-03-16T20:45:31.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcabf7fe640 1 -- 172.21.15.122:0/1520880338 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fcacc0b2040 con 0x7fcad81411e0 2024-03-16T20:45:31.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcabf7fe640 1 --2- 172.21.15.122:0/1520880338 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcac4066e40 0x7fcac4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcabf7fe640 1 -- 172.21.15.122:0/1520880338 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcacc11e340 con 0x7fcad81411e0 2024-03-16T20:45:31.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcad6ffd640 1 --2- 172.21.15.122:0/1520880338 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcac4066e40 0x7fcac4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.599+0000 7fcad6ffd640 1 --2- 172.21.15.122:0/1520880338 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcac4066e40 0x7fcac4069300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcac8000f70 tx=0x7fcac8007e90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:31.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.603+0000 7fcabf7fe640 1 -- 172.21.15.122:0/1520880338 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fcad80b7530 con 0x7fcad81411e0 2024-03-16T20:45:31.789 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.783+0000 7fcadd43b640 1 -- 172.21.15.122:0/1520880338 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcad80b7530 con 0x7fcad81411e0 2024-03-16T20:45:31.790 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.787+0000 7fcabf7fe640 1 -- 172.21.15.122:0/1520880338 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3397 (secure 0 0 0) 0x7fcacc0b0d60 con 0x7fcad81411e0 2024-03-16T20:45:31.790 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:31.790 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3481092793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3481092793},{"type":"v1","addr":"172.21.15.122:6833","nonce":3481092793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T20:45:04.711887+0000","modified":"2024-03-16T20:45:06.765685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/1428608180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1428608180},{"type":"v1","addr":"172.21.15.195:6835","nonce":1428608180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-03-16T20:45:31.794 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 6 2024-03-16T20:45:31.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.787+0000 7fcadd43b640 1 -- 172.21.15.122:0/1520880338 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcac4066e40 msgr2=0x7fcac4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:31.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.787+0000 7fcadd43b640 1 --2- 172.21.15.122:0/1520880338 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcac4066e40 0x7fcac4069300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcac8000f70 tx=0x7fcac8007e90 comp rx=0 tx=0).stop 2024-03-16T20:45:31.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.787+0000 7fcadd43b640 1 -- 172.21.15.122:0/1520880338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcad81411e0 msgr2=0x7fcad81dfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:31.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.787+0000 7fcadd43b640 1 --2- 172.21.15.122:0/1520880338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcad81411e0 0x7fcad81dfbe0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0ae040 tx=0x7fcacc019240 comp rx=0 tx=0).stop 2024-03-16T20:45:31.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.787+0000 7fcadd43b640 1 -- 172.21.15.122:0/1520880338 shutdown_connections 2024-03-16T20:45:31.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.787+0000 7fcadd43b640 1 --2- 172.21.15.122:0/1520880338 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcac4066e40 0x7fcac4069300 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.787+0000 7fcadd43b640 1 --2- 172.21.15.122:0/1520880338 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcad814fc20 0x7fcad8151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.787+0000 7fcadd43b640 1 --2- 172.21.15.122:0/1520880338 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcad81411e0 0x7fcad81dfbe0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.787+0000 7fcadd43b640 1 --2- 172.21.15.122:0/1520880338 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcad806b1f0 0x7fcad814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.787+0000 7fcadd43b640 1 -- 172.21.15.122:0/1520880338 >> 172.21.15.122:0/1520880338 conn(0x7fcad8070d30 msgr2=0x7fcad8071110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:31.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.787+0000 7fcadd43b640 1 -- 172.21.15.122:0/1520880338 shutdown_connections 2024-03-16T20:45:31.795 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.787+0000 7fcadd43b640 1 -- 172.21.15.122:0/1520880338 wait complete. 2024-03-16T20:45:31.808 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:31.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.923+0000 7f56e813e640 1 Processor -- start 2024-03-16T20:45:31.929 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.923+0000 7f56e813e640 1 -- start start 2024-03-16T20:45:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e813e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f56e00b9140 0x7f56e009e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e813e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f56e009c150 0x7f56e009e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e813e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e009edb0 con 0x7f56e006c060 2024-03-16T20:45:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e813e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e009ae50 con 0x7f56e009c150 2024-03-16T20:45:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e813e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e009a800 con 0x7f56e00b9140 2024-03-16T20:45:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e5eda640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f56e006c060 0x7f56e009b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39822/0 (socket says 172.21.15.122:39822) 2024-03-16T20:45:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e5eda640 1 -- 172.21.15.122:0/3295244029 learned_addr learned my addr 172.21.15.122:0/3295244029 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e56d9640 1 --2- 172.21.15.122:0/3295244029 >> v2:172.21.15.195:3300/0 conn(0x7f56e00b9140 0x7f56e009e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e66db640 1 --2- 172.21.15.122:0/3295244029 >> v2:172.21.15.122:3300/0 conn(0x7f56e009c150 0x7f56e009e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e4ed8640 1 -- 172.21.15.122:0/3295244029 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2056090099 0 0) 0x7f56e009edb0 con 0x7f56e006c060 2024-03-16T20:45:31.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e4ed8640 1 -- 172.21.15.122:0/3295244029 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56c8003530 con 0x7f56e006c060 2024-03-16T20:45:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e66db640 1 -- 172.21.15.122:0/3295244029 >> v2:172.21.15.195:3300/0 conn(0x7f56e00b9140 msgr2=0x7f56e009e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e66db640 1 --2- 172.21.15.122:0/3295244029 >> v2:172.21.15.195:3300/0 conn(0x7f56e00b9140 0x7f56e009e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e66db640 1 -- 172.21.15.122:0/3295244029 >> v1:172.21.15.122:6789/0 conn(0x7f56e006c060 legacy=0x7f56e009b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:31.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e66db640 1 -- 172.21.15.122:0/3295244029 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e006c440 con 0x7f56e009c150 2024-03-16T20:45:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e66db640 1 --2- 172.21.15.122:0/3295244029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56e009c150 0x7f56e009e870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f56d00097c0 tx=0x7f56d001cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e595ba5dfbac05b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e4ed8640 1 -- 172.21.15.122:0/3295244029 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d00ab600 con 0x7f56e009c150 2024-03-16T20:45:31.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e4ed8640 1 -- 172.21.15.122:0/3295244029 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d00a7050 con 0x7f56e009c150 2024-03-16T20:45:31.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.927+0000 7f56e4ed8640 1 -- 172.21.15.122:0/3295244029 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d00ab960 con 0x7f56e009c150 2024-03-16T20:45:31.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 -- 172.21.15.122:0/3295244029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56e009c150 msgr2=0x7f56e009e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:31.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 --2- 172.21.15.122:0/3295244029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56e009c150 0x7f56e009e870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f56d00097c0 tx=0x7f56d001cd70 comp rx=0 tx=0).stop 2024-03-16T20:45:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 -- 172.21.15.122:0/3295244029 shutdown_connections 2024-03-16T20:45:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 --2- 172.21.15.122:0/3295244029 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56e009c150 0x7f56e009e870 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 --2- 172.21.15.122:0/3295244029 >> v2:172.21.15.195:3300/0 conn(0x7f56e00b9140 0x7f56e009e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 -- 172.21.15.122:0/3295244029 >> 172.21.15.122:0/3295244029 conn(0x7f56e00bd000 msgr2=0x7f56e00bd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 -- 172.21.15.122:0/3295244029 shutdown_connections 2024-03-16T20:45:31.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 -- 172.21.15.122:0/3295244029 wait complete. 2024-03-16T20:45:31.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 Processor -- start 2024-03-16T20:45:31.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 -- start start 2024-03-16T20:45:31.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56e006c060 0x7f56e00b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56e009c150 0x7f56e00a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56e00b9140 0x7f56e00a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e009fd20 con 0x7f56e006c060 2024-03-16T20:45:31.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e00a1ad0 con 0x7f56e009c150 2024-03-16T20:45:31.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e00a2910 con 0x7f56e00b9140 2024-03-16T20:45:31.940 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e5eda640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56e006c060 0x7f56e00b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e5eda640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56e006c060 0x7f56e00b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34200/0 (socket says 172.21.15.122:34200) 2024-03-16T20:45:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e5eda640 1 -- 172.21.15.122:0/379117944 learned_addr learned my addr 172.21.15.122:0/379117944 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e56d9640 1 --2- 172.21.15.122:0/379117944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56e009c150 0x7f56e00a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e66db640 1 --2- 172.21.15.122:0/379117944 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56e00b9140 0x7f56e00a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e5eda640 1 -- 172.21.15.122:0/379117944 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56e00b9140 msgr2=0x7f56e00a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e5eda640 1 --2- 172.21.15.122:0/379117944 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56e00b9140 0x7f56e00a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e5eda640 1 -- 172.21.15.122:0/379117944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56e009c150 msgr2=0x7f56e00a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e5eda640 1 --2- 172.21.15.122:0/379117944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56e009c150 0x7f56e00a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e5eda640 1 -- 172.21.15.122:0/379117944 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d0009420 con 0x7f56e006c060 2024-03-16T20:45:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e56d9640 1 --2- 172.21.15.122:0/379117944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56e009c150 0x7f56e00a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e5eda640 1 --2- 172.21.15.122:0/379117944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56e006c060 0x7f56e00b3470 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f56d800cb40 tx=0x7f56d800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56de7fc640 1 -- 172.21.15.122:0/379117944 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d800aca0 con 0x7f56e006c060 2024-03-16T20:45:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56de7fc640 1 -- 172.21.15.122:0/379117944 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d800ae00 con 0x7f56e006c060 2024-03-16T20:45:31.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 -- 172.21.15.122:0/379117944 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e014a9b0 con 0x7f56e006c060 2024-03-16T20:45:31.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56de7fc640 1 -- 172.21.15.122:0/379117944 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d8012620 con 0x7f56e006c060 2024-03-16T20:45:31.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.931+0000 7f56e813e640 1 -- 172.21.15.122:0/379117944 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e009ab70 con 0x7f56e006c060 2024-03-16T20:45:31.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.935+0000 7f56e813e640 1 -- 172.21.15.122:0/379117944 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e009fe90 con 0x7f56e006c060 2024-03-16T20:45:31.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.935+0000 7f56de7fc640 1 -- 172.21.15.122:0/379117944 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f56d8007070 con 0x7f56e006c060 2024-03-16T20:45:31.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.935+0000 7f56de7fc640 1 --2- 172.21.15.122:0/379117944 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f56cc066e40 0x7f56cc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:31.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.935+0000 7f56de7fc640 1 -- 172.21.15.122:0/379117944 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56d8084ab0 con 0x7f56e006c060 2024-03-16T20:45:31.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.935+0000 7f56e56d9640 1 --2- 172.21.15.122:0/379117944 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f56cc066e40 0x7f56cc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:31.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.935+0000 7f56e56d9640 1 --2- 172.21.15.122:0/379117944 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f56cc066e40 0x7f56cc069300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f56e00bc7b0 tx=0x7f56d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:31.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:31.939+0000 7f56de7fc640 1 -- 172.21.15.122:0/379117944 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f56e009fe90 con 0x7f56e006c060 2024-03-16T20:45:32.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.099+0000 7f56e813e640 1 -- 172.21.15.122:0/379117944 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56e009fe60 con 0x7f56e006c060 2024-03-16T20:45:32.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.099+0000 7f56de7fc640 1 -- 172.21.15.122:0/379117944 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f56d80553d0 con 0x7f56e006c060 2024-03-16T20:45:32.105 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:32.105 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:06.404479+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.103+0000 7f56e813e640 1 -- 172.21.15.122:0/379117944 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f56cc066e40 msgr2=0x7f56cc069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.103+0000 7f56e813e640 1 --2- 172.21.15.122:0/379117944 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f56cc066e40 0x7f56cc069300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f56e00bc7b0 tx=0x7f56d4008040 comp rx=0 tx=0).stop 2024-03-16T20:45:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.103+0000 7f56e813e640 1 -- 172.21.15.122:0/379117944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56e006c060 msgr2=0x7f56e00b3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.103+0000 7f56e813e640 1 --2- 172.21.15.122:0/379117944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56e006c060 0x7f56e00b3470 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f56d800cb40 tx=0x7f56d800a060 comp rx=0 tx=0).stop 2024-03-16T20:45:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.103+0000 7f56e813e640 1 -- 172.21.15.122:0/379117944 shutdown_connections 2024-03-16T20:45:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.103+0000 7f56e813e640 1 --2- 172.21.15.122:0/379117944 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f56cc066e40 0x7f56cc069300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.103+0000 7f56e813e640 1 --2- 172.21.15.122:0/379117944 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f56e00b9140 0x7f56e00a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.103+0000 7f56e813e640 1 --2- 172.21.15.122:0/379117944 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f56e009c150 0x7f56e00a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.103+0000 7f56e813e640 1 --2- 172.21.15.122:0/379117944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f56e006c060 0x7f56e00b3470 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.103+0000 7f56e813e640 1 -- 172.21.15.122:0/379117944 >> 172.21.15.122:0/379117944 conn(0x7f56e00bd000 msgr2=0x7f56e00bb270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.103+0000 7f56e813e640 1 -- 172.21.15.122:0/379117944 shutdown_connections 2024-03-16T20:45:32.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.103+0000 7f56e813e640 1 -- 172.21.15.122:0/379117944 wait complete. 2024-03-16T20:45:32.125 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T20:44:52.653523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T20:45:02.758515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T20:45:03.710783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T20:45:32.125 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T20:45:32.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.235+0000 7f50b13ef640 1 Processor -- start 2024-03-16T20:45:32.241 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.235+0000 7f50b13ef640 1 -- start start 2024-03-16T20:45:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50b13ef640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50ac141580 0x7f50ac0baac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50b13ef640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ac147ff0 con 0x7f50ac06d350 2024-03-16T20:45:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50b13ef640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ac0bb460 con 0x7f50ac0bb000 2024-03-16T20:45:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50b13ef640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac1479a0 con 0x7f50ac141580 2024-03-16T20:45:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50ab7fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f50ac06d350 0x7f50ac151920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39828/0 (socket says 172.21.15.122:39828) 2024-03-16T20:45:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50ab7fe640 1 -- 172.21.15.122:0/2925493662 learned_addr learned my addr 172.21.15.122:0/2925493662 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aaffd640 1 --2- 172.21.15.122:0/2925493662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50ac141580 0x7f50ac0baac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aa7fc640 1 -- 172.21.15.122:0/2925493662 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839108508 0 0) 0x7f50ac147ff0 con 0x7f50ac06d350 2024-03-16T20:45:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aa7fc640 1 -- 172.21.15.122:0/2925493662 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5098003530 con 0x7f50ac06d350 2024-03-16T20:45:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aa7fc640 1 -- 172.21.15.122:0/2925493662 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 709594112 0 0) 0x7f5098003530 con 0x7f50ac06d350 2024-03-16T20:45:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aa7fc640 1 -- 172.21.15.122:0/2925493662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50ac141580 msgr2=0x7f50ac0baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aa7fc640 1 --2- 172.21.15.122:0/2925493662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50ac141580 0x7f50ac0baac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aa7fc640 1 -- 172.21.15.122:0/2925493662 >> v1:172.21.15.195:6789/0 conn(0x7f50ac0bb000 legacy=0x7f50ac06cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aa7fc640 1 -- 172.21.15.122:0/2925493662 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac147c80 con 0x7f50ac06d350 2024-03-16T20:45:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aa7fc640 1 -- 172.21.15.122:0/2925493662 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f50a4001a00 con 0x7f50ac06d350 2024-03-16T20:45:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aa7fc640 1 -- 172.21.15.122:0/2925493662 >> v1:172.21.15.122:6789/0 conn(0x7f50ac06d350 legacy=0x7f50ac151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aa7fc640 1 --2- 172.21.15.122:0/2925493662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50980039b0 0x7f5098003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aa7fc640 1 --2- 172.21.15.122:0/2925493662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50ac141580 0x7f50980047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aa7fc640 1 --2- 172.21.15.122:0/2925493662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5098004d10 0x7f5098005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aa7fc640 1 -- 172.21.15.122:0/2925493662 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac1479a0 con 0x7f50980039b0 2024-03-16T20:45:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aa7fc640 1 -- 172.21.15.122:0/2925493662 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509800b990 con 0x7f5098004d10 2024-03-16T20:45:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aa7fc640 1 -- 172.21.15.122:0/2925493662 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509800bb50 con 0x7f50ac141580 2024-03-16T20:45:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aaffd640 1 --2- 172.21.15.122:0/2925493662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50980039b0 0x7f5098003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50ab7fe640 1 --2- 172.21.15.122:0/2925493662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5098004d10 0x7f5098005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50a27fc640 1 --2- 172.21.15.122:0/2925493662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50ac141580 0x7f50980047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aaffd640 1 -- 172.21.15.122:0/2925493662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50ac141580 msgr2=0x7f50980047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aaffd640 1 --2- 172.21.15.122:0/2925493662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50ac141580 0x7f50980047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aaffd640 1 -- 172.21.15.122:0/2925493662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5098004d10 msgr2=0x7f5098005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:32.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aaffd640 1 --2- 172.21.15.122:0/2925493662 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5098004d10 0x7f5098005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aaffd640 1 -- 172.21.15.122:0/2925493662 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5098003530 con 0x7f50980039b0 2024-03-16T20:45:32.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aaffd640 1 --2- 172.21.15.122:0/2925493662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50980039b0 0x7f5098003d90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f509c009ae0 tx=0x7f509c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4d84a2d747d8058 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:32.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aa7fc640 1 -- 172.21.15.122:0/2925493662 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c0a55a0 con 0x7f50980039b0 2024-03-16T20:45:32.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.239+0000 7f50aa7fc640 1 -- 172.21.15.122:0/2925493662 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c0a5700 con 0x7f50980039b0 2024-03-16T20:45:32.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50aa7fc640 1 -- 172.21.15.122:0/2925493662 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c0a5ab0 con 0x7f50980039b0 2024-03-16T20:45:32.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50b13ef640 1 -- 172.21.15.122:0/2925493662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50980039b0 msgr2=0x7f5098003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50b13ef640 1 --2- 172.21.15.122:0/2925493662 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50980039b0 0x7f5098003d90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f509c009ae0 tx=0x7f509c01c9b0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50ab7fe640 1 -- 172.21.15.122:0/2925493662 reap_dead start 2024-03-16T20:45:32.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50b13ef640 1 -- 172.21.15.122:0/2925493662 shutdown_connections 2024-03-16T20:45:32.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50b13ef640 1 -- 172.21.15.122:0/2925493662 >> 172.21.15.122:0/2925493662 conn(0x7f50ac0bca20 msgr2=0x7f50ac0bce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:32.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50b13ef640 1 -- 172.21.15.122:0/2925493662 shutdown_connections 2024-03-16T20:45:32.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50b13ef640 1 -- 172.21.15.122:0/2925493662 wait complete. 2024-03-16T20:45:32.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50b13ef640 1 Processor -- start 2024-03-16T20:45:32.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50b13ef640 1 -- start start 2024-03-16T20:45:32.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50b13ef640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50980039b0 0x7f50ac0b6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50b13ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50ac141a40 0x7f50ac0b5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50b13ef640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50ac1427e0 0x7f50ac1e47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50b13ef640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac0b4870 con 0x7f50ac141a40 2024-03-16T20:45:32.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50b13ef640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac13dcd0 con 0x7f50980039b0 2024-03-16T20:45:32.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50b13ef640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac0b3d20 con 0x7f50ac1427e0 2024-03-16T20:45:32.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50a27fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50ac141a40 0x7f50ac0b5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50a27fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50ac141a40 0x7f50ac0b5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34218/0 (socket says 172.21.15.122:34218) 2024-03-16T20:45:32.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50a27fc640 1 -- 172.21.15.122:0/2217526722 learned_addr learned my addr 172.21.15.122:0/2217526722 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:32.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50ab7fe640 1 --2- 172.21.15.122:0/2217526722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50ac1427e0 0x7f50ac1e47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50aaffd640 1 --2- 172.21.15.122:0/2217526722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50980039b0 0x7f50ac0b6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50a27fc640 1 -- 172.21.15.122:0/2217526722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50ac1427e0 msgr2=0x7f50ac1e47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50a27fc640 1 --2- 172.21.15.122:0/2217526722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50ac1427e0 0x7f50ac1e47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50a27fc640 1 -- 172.21.15.122:0/2217526722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50980039b0 msgr2=0x7f50ac0b6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50a27fc640 1 --2- 172.21.15.122:0/2217526722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50980039b0 0x7f50ac0b6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50a27fc640 1 -- 172.21.15.122:0/2217526722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac0b4a50 con 0x7f50ac141a40 2024-03-16T20:45:32.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50aaffd640 1 --2- 172.21.15.122:0/2217526722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50980039b0 0x7f50ac0b6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:32.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50a27fc640 1 --2- 172.21.15.122:0/2217526722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50ac141a40 0x7f50ac0b5970 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5094010f70 tx=0x7f509400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:32.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50a3fff640 1 -- 172.21.15.122:0/2217526722 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094005070 con 0x7f50ac141a40 2024-03-16T20:45:32.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50a3fff640 1 -- 172.21.15.122:0/2217526722 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509400d030 con 0x7f50ac141a40 2024-03-16T20:45:32.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50a3fff640 1 -- 172.21.15.122:0/2217526722 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5094005430 con 0x7f50ac141a40 2024-03-16T20:45:32.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50b13ef640 1 -- 172.21.15.122:0/2217526722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ac13e0f0 con 0x7f50ac141a40 2024-03-16T20:45:32.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.243+0000 7f50b13ef640 1 -- 172.21.15.122:0/2217526722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac142d90 con 0x7f50ac141a40 2024-03-16T20:45:32.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.247+0000 7f50b13ef640 1 -- 172.21.15.122:0/2217526722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ac144d40 con 0x7f50ac141a40 2024-03-16T20:45:32.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.247+0000 7f50a3fff640 1 -- 172.21.15.122:0/2217526722 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f5094003600 con 0x7f50ac141a40 2024-03-16T20:45:32.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.247+0000 7f50a3fff640 1 --2- 172.21.15.122:0/2217526722 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f508c066e70 0x7f508c069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.247+0000 7f50a3fff640 1 -- 172.21.15.122:0/2217526722 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5094088220 con 0x7f50ac141a40 2024-03-16T20:45:32.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.247+0000 7f50aaffd640 1 --2- 172.21.15.122:0/2217526722 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f508c066e70 0x7f508c069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.247+0000 7f50aaffd640 1 --2- 172.21.15.122:0/2217526722 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f508c066e70 0x7f508c069330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f509c0af000 tx=0x7f509c0046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:32.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.251+0000 7f50a3fff640 1 -- 172.21.15.122:0/2217526722 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f50940550a0 con 0x7f50ac141a40 2024-03-16T20:45:32.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.399+0000 7f50b13ef640 1 -- 172.21.15.122:0/2217526722 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f50ac0684d0 con 0x7f50ac141a40 2024-03-16T20:45:32.403 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.399+0000 7f50a3fff640 1 -- 172.21.15.122:0/2217526722 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5094058760 con 0x7f50ac141a40 2024-03-16T20:45:32.403 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-03-16T20:45:32.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.399+0000 7f50b13ef640 1 -- 172.21.15.122:0/2217526722 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f508c066e70 msgr2=0x7f508c069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.399+0000 7f50b13ef640 1 --2- 172.21.15.122:0/2217526722 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f508c066e70 0x7f508c069330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f509c0af000 tx=0x7f509c0046d0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.399+0000 7f50b13ef640 1 -- 172.21.15.122:0/2217526722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50ac141a40 msgr2=0x7f50ac0b5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.399+0000 7f50b13ef640 1 --2- 172.21.15.122:0/2217526722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50ac141a40 0x7f50ac0b5970 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5094010f70 tx=0x7f509400e530 comp rx=0 tx=0).stop 2024-03-16T20:45:32.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.399+0000 7f50b13ef640 1 -- 172.21.15.122:0/2217526722 shutdown_connections 2024-03-16T20:45:32.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.399+0000 7f50b13ef640 1 --2- 172.21.15.122:0/2217526722 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f508c066e70 0x7f508c069330 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.399+0000 7f50b13ef640 1 --2- 172.21.15.122:0/2217526722 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50ac1427e0 0x7f50ac1e47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.399+0000 7f50b13ef640 1 --2- 172.21.15.122:0/2217526722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f50ac141a40 0x7f50ac0b5970 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.399+0000 7f50b13ef640 1 --2- 172.21.15.122:0/2217526722 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50980039b0 0x7f50ac0b6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.399+0000 7f50b13ef640 1 -- 172.21.15.122:0/2217526722 >> 172.21.15.122:0/2217526722 conn(0x7f50ac0bca20 msgr2=0x7f50ac0b2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:32.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.399+0000 7f50b13ef640 1 -- 172.21.15.122:0/2217526722 shutdown_connections 2024-03-16T20:45:32.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.399+0000 7f50b13ef640 1 -- 172.21.15.122:0/2217526722 wait complete. 2024-03-16T20:45:32.421 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T20:45:32.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e78cd640 1 Processor -- start 2024-03-16T20:45:32.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e78cd640 1 -- start start 2024-03-16T20:45:32.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e78cd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f76e009ae30 0x7f76e00b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e78cd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76e00b7ba0 0x7f76e00b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e78cd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f76e01422e0 0x7f76e0142730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e78cd640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e00b50a0 con 0x7f76e009ae30 2024-03-16T20:45:32.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e78cd640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76e00b8560 con 0x7f76e01422e0 2024-03-16T20:45:32.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e78cd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00b3fa0 con 0x7f76e00b7ba0 2024-03-16T20:45:32.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e5669640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f76e009ae30 0x7f76e00b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e4e68640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76e00b7ba0 0x7f76e00b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e5e6a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f76e01422e0 0x7f76e0142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e5669640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f76e009ae30 0x7f76e00b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34224/0 (socket says 172.21.15.122:34224) 2024-03-16T20:45:32.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e4e68640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76e00b7ba0 0x7f76e00b8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:50258/0 (socket says 172.21.15.122:50258) 2024-03-16T20:45:32.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e5669640 1 -- 172.21.15.122:0/1942434912 learned_addr learned my addr 172.21.15.122:0/1942434912 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:32.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e5669640 1 -- 172.21.15.122:0/1942434912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76e00b7ba0 msgr2=0x7f76e00b8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:32.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e5669640 1 --2- 172.21.15.122:0/1942434912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76e00b7ba0 0x7f76e00b8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e5669640 1 -- 172.21.15.122:0/1942434912 >> v2:172.21.15.195:3300/0 conn(0x7f76e01422e0 msgr2=0x7f76e0142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:32.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e5669640 1 --2- 172.21.15.122:0/1942434912 >> v2:172.21.15.195:3300/0 conn(0x7f76e01422e0 0x7f76e0142730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.535+0000 7f76e5669640 1 -- 172.21.15.122:0/1942434912 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e009b210 con 0x7f76e009ae30 2024-03-16T20:45:32.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.539+0000 7f76e5669640 1 --2- 172.21.15.122:0/1942434912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76e009ae30 0x7f76e00b4430 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f76e009b400 tx=0x7f76d001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380610204c0207a9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:32.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.539+0000 7f76d7fff640 1 -- 172.21.15.122:0/1942434912 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d00aa640 con 0x7f76e009ae30 2024-03-16T20:45:32.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.539+0000 7f76d7fff640 1 -- 172.21.15.122:0/1942434912 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d00aa7a0 con 0x7f76e009ae30 2024-03-16T20:45:32.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.539+0000 7f76d7fff640 1 -- 172.21.15.122:0/1942434912 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d00aab30 con 0x7f76e009ae30 2024-03-16T20:45:32.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 -- 172.21.15.122:0/1942434912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76e009ae30 msgr2=0x7f76e00b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 --2- 172.21.15.122:0/1942434912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76e009ae30 0x7f76e00b4430 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f76e009b400 tx=0x7f76d001c520 comp rx=0 tx=0).stop 2024-03-16T20:45:32.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 -- 172.21.15.122:0/1942434912 shutdown_connections 2024-03-16T20:45:32.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 --2- 172.21.15.122:0/1942434912 >> v2:172.21.15.195:3300/0 conn(0x7f76e01422e0 0x7f76e0142730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 --2- 172.21.15.122:0/1942434912 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76e00b7ba0 0x7f76e00b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 --2- 172.21.15.122:0/1942434912 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76e009ae30 0x7f76e00b4430 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 -- 172.21.15.122:0/1942434912 >> 172.21.15.122:0/1942434912 conn(0x7f76e00b8d10 msgr2=0x7f76e009a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:32.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 -- 172.21.15.122:0/1942434912 shutdown_connections 2024-03-16T20:45:32.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 -- 172.21.15.122:0/1942434912 wait complete. 2024-03-16T20:45:32.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 Processor -- start 2024-03-16T20:45:32.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 -- start start 2024-03-16T20:45:32.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76e009ae30 0x7f76e006fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76e00b7ba0 0x7f76e0072280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76e01422e0 0x7f76e013c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e00714a0 con 0x7f76e01422e0 2024-03-16T20:45:32.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76e0144d20 con 0x7f76e00b7ba0 2024-03-16T20:45:32.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76e0145000 con 0x7f76e009ae30 2024-03-16T20:45:32.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e5e6a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76e01422e0 0x7f76e013c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e5e6a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76e01422e0 0x7f76e013c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34236/0 (socket says 172.21.15.122:34236) 2024-03-16T20:45:32.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e5e6a640 1 -- 172.21.15.122:0/2836586060 learned_addr learned my addr 172.21.15.122:0/2836586060 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:32.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e5669640 1 --2- 172.21.15.122:0/2836586060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76e009ae30 0x7f76e006fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e4e68640 1 --2- 172.21.15.122:0/2836586060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76e00b7ba0 0x7f76e0072280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e5e6a640 1 -- 172.21.15.122:0/2836586060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76e009ae30 msgr2=0x7f76e006fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e5e6a640 1 --2- 172.21.15.122:0/2836586060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76e009ae30 0x7f76e006fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e5e6a640 1 -- 172.21.15.122:0/2836586060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76e00b7ba0 msgr2=0x7f76e0072280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e5e6a640 1 --2- 172.21.15.122:0/2836586060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76e00b7ba0 0x7f76e0072280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e5e6a640 1 -- 172.21.15.122:0/2836586060 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d00094d0 con 0x7f76e01422e0 2024-03-16T20:45:32.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e4e68640 1 --2- 172.21.15.122:0/2836586060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76e00b7ba0 0x7f76e0072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:32.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e5e6a640 1 --2- 172.21.15.122:0/2836586060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76e01422e0 0x7f76e013c4b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f76d800efc0 tx=0x7f76d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:32.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76d5ffb640 1 -- 172.21.15.122:0/2836586060 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d8009ba0 con 0x7f76e01422e0 2024-03-16T20:45:32.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76d5ffb640 1 -- 172.21.15.122:0/2836586060 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d800f040 con 0x7f76e01422e0 2024-03-16T20:45:32.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76d5ffb640 1 -- 172.21.15.122:0/2836586060 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d80146c0 con 0x7f76e01422e0 2024-03-16T20:45:32.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 -- 172.21.15.122:0/2836586060 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76e0143790 con 0x7f76e01422e0 2024-03-16T20:45:32.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.543+0000 7f76e78cd640 1 -- 172.21.15.122:0/2836586060 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76e0072ae0 con 0x7f76e01422e0 2024-03-16T20:45:32.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.547+0000 7f76e78cd640 1 -- 172.21.15.122:0/2836586060 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76e0074bb0 con 0x7f76e01422e0 2024-03-16T20:45:32.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.547+0000 7f76d5ffb640 1 -- 172.21.15.122:0/2836586060 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f76d8014820 con 0x7f76e01422e0 2024-03-16T20:45:32.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.547+0000 7f76d5ffb640 1 --2- 172.21.15.122:0/2836586060 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f76c4066e40 0x7f76c4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.547+0000 7f76d5ffb640 1 -- 172.21.15.122:0/2836586060 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76d8085b90 con 0x7f76e01422e0 2024-03-16T20:45:32.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.547+0000 7f76e5669640 1 --2- 172.21.15.122:0/2836586060 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f76c4066e40 0x7f76c4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.547+0000 7f76e5669640 1 --2- 172.21.15.122:0/2836586060 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f76c4066e40 0x7f76c4069300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f76d00ae000 tx=0x7f76d00a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:32.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.551+0000 7f76d5ffb640 1 -- 172.21.15.122:0/2836586060 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f76d8052af0 con 0x7f76e01422e0 2024-03-16T20:45:32.703 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.699+0000 7f76e78cd640 1 -- 172.21.15.122:0/2836586060 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f76e009a860 con 0x7f76e01422e0 2024-03-16T20:45:32.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.699+0000 7f76d5ffb640 1 -- 172.21.15.122:0/2836586060 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f76d80180a0 con 0x7f76e01422e0 2024-03-16T20:45:32.704 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-03-16T20:45:32.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.703+0000 7f76e78cd640 1 -- 172.21.15.122:0/2836586060 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f76c4066e40 msgr2=0x7f76c4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.703+0000 7f76e78cd640 1 --2- 172.21.15.122:0/2836586060 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f76c4066e40 0x7f76c4069300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f76d00ae000 tx=0x7f76d00a7000 comp rx=0 tx=0).stop 2024-03-16T20:45:32.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.703+0000 7f76e78cd640 1 -- 172.21.15.122:0/2836586060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76e01422e0 msgr2=0x7f76e013c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.703+0000 7f76e78cd640 1 --2- 172.21.15.122:0/2836586060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76e01422e0 0x7f76e013c4b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f76d800efc0 tx=0x7f76d800c490 comp rx=0 tx=0).stop 2024-03-16T20:45:32.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.703+0000 7f76e78cd640 1 -- 172.21.15.122:0/2836586060 shutdown_connections 2024-03-16T20:45:32.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.703+0000 7f76e78cd640 1 --2- 172.21.15.122:0/2836586060 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f76c4066e40 0x7f76c4069300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.703+0000 7f76e78cd640 1 --2- 172.21.15.122:0/2836586060 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f76e01422e0 0x7f76e013c4b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.703+0000 7f76e78cd640 1 --2- 172.21.15.122:0/2836586060 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f76e00b7ba0 0x7f76e0072280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.703+0000 7f76e78cd640 1 --2- 172.21.15.122:0/2836586060 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f76e009ae30 0x7f76e006fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.703+0000 7f76e78cd640 1 -- 172.21.15.122:0/2836586060 >> 172.21.15.122:0/2836586060 conn(0x7f76e00b8d10 msgr2=0x7f76e0141730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:32.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.703+0000 7f76e78cd640 1 -- 172.21.15.122:0/2836586060 shutdown_connections 2024-03-16T20:45:32.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.703+0000 7f76e78cd640 1 -- 172.21.15.122:0/2836586060 wait complete. 2024-03-16T20:45:32.721 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T20:45:32.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.827+0000 7fd8b8403640 1 Processor -- start 2024-03-16T20:45:32.833 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.827+0000 7fd8b8403640 1 -- start start 2024-03-16T20:45:32.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b8403640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd8b009ae30 0x7fd8b00b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b8403640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd8b0144aa0 0x7fd8b014e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b8403640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b009b210 con 0x7fd8b00aef80 2024-03-16T20:45:32.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b8403640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b00b3110 con 0x7fd8b009ae30 2024-03-16T20:45:32.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b8403640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b00b3970 con 0x7fd8b0144aa0 2024-03-16T20:45:32.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b619f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd8b009ae30 0x7fd8b00b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b619f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd8b009ae30 0x7fd8b00b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34248/0 (socket says 172.21.15.122:34248) 2024-03-16T20:45:32.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b619f640 1 -- 172.21.15.122:0/3006253145 learned_addr learned my addr 172.21.15.122:0/3006253145 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:32.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b69a0640 1 --2- 172.21.15.122:0/3006253145 >> v2:172.21.15.195:3300/0 conn(0x7fd8b0144aa0 0x7fd8b014e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b619f640 1 -- 172.21.15.122:0/3006253145 >> v2:172.21.15.195:3300/0 conn(0x7fd8b0144aa0 msgr2=0x7fd8b014e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:32.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b619f640 1 --2- 172.21.15.122:0/3006253145 >> v2:172.21.15.195:3300/0 conn(0x7fd8b0144aa0 0x7fd8b014e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b619f640 1 -- 172.21.15.122:0/3006253145 >> v1:172.21.15.195:6789/0 conn(0x7fd8b00aef80 legacy=0x7fd8b0144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:32.835 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b619f640 1 -- 172.21.15.122:0/3006253145 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b00b3320 con 0x7fd8b009ae30 2024-03-16T20:45:32.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b619f640 1 --2- 172.21.15.122:0/3006253145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b009ae30 0x7fd8b00b3b60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00b7720 tx=0x7fd8a001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b00b96e4d360a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:32.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b519d640 1 -- 172.21.15.122:0/3006253145 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00ad660 con 0x7fd8b009ae30 2024-03-16T20:45:32.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b519d640 1 -- 172.21.15.122:0/3006253145 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a00ad7c0 con 0x7fd8b009ae30 2024-03-16T20:45:32.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b519d640 1 -- 172.21.15.122:0/3006253145 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00adb50 con 0x7fd8b009ae30 2024-03-16T20:45:32.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b8403640 1 -- 172.21.15.122:0/3006253145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b009ae30 msgr2=0x7fd8b00b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b8403640 1 --2- 172.21.15.122:0/3006253145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b009ae30 0x7fd8b00b3b60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd8b00b7720 tx=0x7fd8a001c450 comp rx=0 tx=0).stop 2024-03-16T20:45:32.837 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b8403640 1 -- 172.21.15.122:0/3006253145 shutdown_connections 2024-03-16T20:45:32.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b8403640 1 --2- 172.21.15.122:0/3006253145 >> v2:172.21.15.195:3300/0 conn(0x7fd8b0144aa0 0x7fd8b014e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b8403640 1 --2- 172.21.15.122:0/3006253145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b009ae30 0x7fd8b00b3b60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b8403640 1 -- 172.21.15.122:0/3006253145 >> 172.21.15.122:0/3006253145 conn(0x7fd8b00b8d10 msgr2=0x7fd8b009a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:32.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b8403640 1 -- 172.21.15.122:0/3006253145 shutdown_connections 2024-03-16T20:45:32.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.831+0000 7fd8b8403640 1 -- 172.21.15.122:0/3006253145 wait complete. 2024-03-16T20:45:32.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b8403640 1 Processor -- start 2024-03-16T20:45:32.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b8403640 1 -- start start 2024-03-16T20:45:32.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b8403640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b009ae30 0x7fd8b0143ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b8403640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8b00aef80 0x7fd8b013bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b8403640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8b0144aa0 0x7fd8b01e10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b8403640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00b0af0 con 0x7fd8b009ae30 2024-03-16T20:45:32.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b8403640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b00aff80 con 0x7fd8b00aef80 2024-03-16T20:45:32.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b8403640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0142cd0 con 0x7fd8b0144aa0 2024-03-16T20:45:32.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b619f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b009ae30 0x7fd8b0143ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b619f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b009ae30 0x7fd8b0143ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34264/0 (socket says 172.21.15.122:34264) 2024-03-16T20:45:32.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b619f640 1 -- 172.21.15.122:0/2015026199 learned_addr learned my addr 172.21.15.122:0/2015026199 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:32.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b599e640 1 --2- 172.21.15.122:0/2015026199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8b00aef80 0x7fd8b013bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b69a0640 1 --2- 172.21.15.122:0/2015026199 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8b0144aa0 0x7fd8b01e10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b619f640 1 -- 172.21.15.122:0/2015026199 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8b0144aa0 msgr2=0x7fd8b01e10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b619f640 1 --2- 172.21.15.122:0/2015026199 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8b0144aa0 0x7fd8b01e10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b619f640 1 -- 172.21.15.122:0/2015026199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8b00aef80 msgr2=0x7fd8b013bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b619f640 1 --2- 172.21.15.122:0/2015026199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8b00aef80 0x7fd8b013bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b619f640 1 -- 172.21.15.122:0/2015026199 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a00094d0 con 0x7fd8b009ae30 2024-03-16T20:45:32.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b599e640 1 --2- 172.21.15.122:0/2015026199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8b00aef80 0x7fd8b013bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:32.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b619f640 1 --2- 172.21.15.122:0/2015026199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b009ae30 0x7fd8b0143ab0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd8a001a080 tx=0x7fd8a00191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:32.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8a6ffd640 1 -- 172.21.15.122:0/2015026199 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00b6640 con 0x7fd8b009ae30 2024-03-16T20:45:32.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8a6ffd640 1 -- 172.21.15.122:0/2015026199 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a00b67a0 con 0x7fd8b009ae30 2024-03-16T20:45:32.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b8403640 1 -- 172.21.15.122:0/2015026199 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b0142f70 con 0x7fd8b009ae30 2024-03-16T20:45:32.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8a6ffd640 1 -- 172.21.15.122:0/2015026199 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a00b6b10 con 0x7fd8b009ae30 2024-03-16T20:45:32.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b8403640 1 -- 172.21.15.122:0/2015026199 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b0141e90 con 0x7fd8b009ae30 2024-03-16T20:45:32.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8b8403640 1 -- 172.21.15.122:0/2015026199 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b0142b30 con 0x7fd8b009ae30 2024-03-16T20:45:32.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.835+0000 7fd8a6ffd640 1 -- 172.21.15.122:0/2015026199 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fd8a00b4070 con 0x7fd8b009ae30 2024-03-16T20:45:32.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.839+0000 7fd8a6ffd640 1 --2- 172.21.15.122:0/2015026199 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd894066e40 0x7fd894069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:32.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.839+0000 7fd8a6ffd640 1 -- 172.21.15.122:0/2015026199 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8a011ef80 con 0x7fd8b009ae30 2024-03-16T20:45:32.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.839+0000 7fd8b599e640 1 --2- 172.21.15.122:0/2015026199 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd894066e40 0x7fd894069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:32.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.839+0000 7fd8b599e640 1 --2- 172.21.15.122:0/2015026199 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd894066e40 0x7fd894069300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0144570 tx=0x7fd8ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:32.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.839+0000 7fd8a6ffd640 1 -- 172.21.15.122:0/2015026199 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd8a00ebde0 con 0x7fd8b009ae30 2024-03-16T20:45:32.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.983+0000 7fd8b8403640 1 -- 172.21.15.122:0/2015026199 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd8b0142b30 con 0x7fd8b009ae30 2024-03-16T20:45:32.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.987+0000 7fd8a6ffd640 1 -- 172.21.15.122:0/2015026199 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd8b0142b30 con 0x7fd8b009ae30 2024-03-16T20:45:32.991 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-03-16T20:45:32.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.987+0000 7fd8b8403640 1 -- 172.21.15.122:0/2015026199 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd894066e40 msgr2=0x7fd894069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.987+0000 7fd8b8403640 1 --2- 172.21.15.122:0/2015026199 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd894066e40 0x7fd894069300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0144570 tx=0x7fd8ac009040 comp rx=0 tx=0).stop 2024-03-16T20:45:32.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.987+0000 7fd8b8403640 1 -- 172.21.15.122:0/2015026199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b009ae30 msgr2=0x7fd8b0143ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:32.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.987+0000 7fd8b8403640 1 --2- 172.21.15.122:0/2015026199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b009ae30 0x7fd8b0143ab0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd8a001a080 tx=0x7fd8a00191f0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.987+0000 7fd8b8403640 1 -- 172.21.15.122:0/2015026199 shutdown_connections 2024-03-16T20:45:32.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.987+0000 7fd8b8403640 1 --2- 172.21.15.122:0/2015026199 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd894066e40 0x7fd894069300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.987+0000 7fd8b8403640 1 --2- 172.21.15.122:0/2015026199 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8b0144aa0 0x7fd8b01e10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.987+0000 7fd8b8403640 1 --2- 172.21.15.122:0/2015026199 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8b00aef80 0x7fd8b013bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.987+0000 7fd8b8403640 1 --2- 172.21.15.122:0/2015026199 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8b009ae30 0x7fd8b0143ab0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:32.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.987+0000 7fd8b8403640 1 -- 172.21.15.122:0/2015026199 >> 172.21.15.122:0/2015026199 conn(0x7fd8b00b8d10 msgr2=0x7fd8b00b5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:32.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.987+0000 7fd8b8403640 1 -- 172.21.15.122:0/2015026199 shutdown_connections 2024-03-16T20:45:32.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:32.987+0000 7fd8b8403640 1 -- 172.21.15.122:0/2015026199 wait complete. 2024-03-16T20:45:33.008 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T20:45:33.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.107+0000 7f8dcca59640 1 Processor -- start 2024-03-16T20:45:33.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.107+0000 7f8dcca59640 1 -- start start 2024-03-16T20:45:33.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dcca59640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f8dc806b820 0x7f8dc813d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dcca59640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8dc8146a50 0x7f8dc8143460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dcca59640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc8147530 con 0x7f8dc813d980 2024-03-16T20:45:33.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dcca59640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc8072600 con 0x7f8dc806b820 2024-03-16T20:45:33.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dcca59640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc80b4c50 con 0x7f8dc8146a50 2024-03-16T20:45:33.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc6d9d640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f8dc813d980 0x7f8dc80b3090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39832/0 (socket says 172.21.15.122:39832) 2024-03-16T20:45:33.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc6d9d640 1 -- 172.21.15.122:0/3492107942 learned_addr learned my addr 172.21.15.122:0/3492107942 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:33.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc5d9b640 1 --2- 172.21.15.122:0/3492107942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8dc8146a50 0x7f8dc8143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc659c640 1 --2- 172.21.15.122:0/3492107942 >> v2:172.21.15.195:3300/0 conn(0x7f8dc806b820 0x7f8dc813d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc559a640 1 -- 172.21.15.122:0/3492107942 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 814524907 0 0) 0x7f8dc8147530 con 0x7f8dc813d980 2024-03-16T20:45:33.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc559a640 1 -- 172.21.15.122:0/3492107942 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8db0003530 con 0x7f8dc813d980 2024-03-16T20:45:33.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc559a640 1 -- 172.21.15.122:0/3492107942 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4033256362 0 0) 0x7f8db0003530 con 0x7f8dc813d980 2024-03-16T20:45:33.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc559a640 1 -- 172.21.15.122:0/3492107942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8dc8146a50 msgr2=0x7f8dc8143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:33.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc559a640 1 --2- 172.21.15.122:0/3492107942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8dc8146a50 0x7f8dc8143460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc559a640 1 -- 172.21.15.122:0/3492107942 >> v2:172.21.15.195:3300/0 conn(0x7f8dc806b820 msgr2=0x7f8dc813d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:33.117 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc559a640 1 --2- 172.21.15.122:0/3492107942 >> v2:172.21.15.195:3300/0 conn(0x7f8dc806b820 0x7f8dc813d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc559a640 1 -- 172.21.15.122:0/3492107942 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc8146ee0 con 0x7f8dc813d980 2024-03-16T20:45:33.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc559a640 1 -- 172.21.15.122:0/3492107942 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f8dc0001680 con 0x7f8dc813d980 2024-03-16T20:45:33.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc559a640 1 -- 172.21.15.122:0/3492107942 >> v1:172.21.15.122:6789/0 conn(0x7f8dc813d980 legacy=0x7f8dc80b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc559a640 1 --2- 172.21.15.122:0/3492107942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8db0003980 0x7f8db0003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc559a640 1 --2- 172.21.15.122:0/3492107942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8db00042e0 0x7f8db00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc559a640 1 --2- 172.21.15.122:0/3492107942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8db0004e20 0x7f8db00052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc559a640 1 -- 172.21.15.122:0/3492107942 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc8072600 con 0x7f8db0004e20 2024-03-16T20:45:33.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc559a640 1 -- 172.21.15.122:0/3492107942 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc80b4c50 con 0x7f8db00042e0 2024-03-16T20:45:33.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc559a640 1 -- 172.21.15.122:0/3492107942 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db000bd10 con 0x7f8db0003980 2024-03-16T20:45:33.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc6d9d640 1 --2- 172.21.15.122:0/3492107942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8db0004e20 0x7f8db00052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc5d9b640 1 --2- 172.21.15.122:0/3492107942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8db00042e0 0x7f8db00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc659c640 1 --2- 172.21.15.122:0/3492107942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8db0003980 0x7f8db0003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc6d9d640 1 -- 172.21.15.122:0/3492107942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8db0003980 msgr2=0x7f8db0003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:33.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc6d9d640 1 --2- 172.21.15.122:0/3492107942 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8db0003980 0x7f8db0003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc6d9d640 1 -- 172.21.15.122:0/3492107942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8db00042e0 msgr2=0x7f8db00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:33.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc6d9d640 1 --2- 172.21.15.122:0/3492107942 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8db00042e0 0x7f8db00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.111+0000 7f8dc6d9d640 1 -- 172.21.15.122:0/3492107942 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db0003530 con 0x7f8db0004e20 2024-03-16T20:45:33.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dc6d9d640 1 --2- 172.21.15.122:0/3492107942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8db0004e20 0x7f8db00052a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000c210 tx=0x7f8dc001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c94ed6e38a2fcb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:33.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dc559a640 1 -- 172.21.15.122:0/3492107942 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc00a5960 con 0x7f8db0004e20 2024-03-16T20:45:33.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dc559a640 1 -- 172.21.15.122:0/3492107942 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc00a5ac0 con 0x7f8db0004e20 2024-03-16T20:45:33.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dc559a640 1 -- 172.21.15.122:0/3492107942 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc00058e0 con 0x7f8db0004e20 2024-03-16T20:45:33.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dcca59640 1 -- 172.21.15.122:0/3492107942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8db0004e20 msgr2=0x7f8db00052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dcca59640 1 --2- 172.21.15.122:0/3492107942 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8db0004e20 0x7f8db00052a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000c210 tx=0x7f8dc001cc60 comp rx=0 tx=0).stop 2024-03-16T20:45:33.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dc6d9d640 1 -- 172.21.15.122:0/3492107942 reap_dead start 2024-03-16T20:45:33.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dcca59640 1 -- 172.21.15.122:0/3492107942 shutdown_connections 2024-03-16T20:45:33.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dcca59640 1 -- 172.21.15.122:0/3492107942 >> 172.21.15.122:0/3492107942 conn(0x7f8dc80b8860 msgr2=0x7f8dc80b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:33.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dcca59640 1 -- 172.21.15.122:0/3492107942 shutdown_connections 2024-03-16T20:45:33.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dcca59640 1 -- 172.21.15.122:0/3492107942 wait complete. 2024-03-16T20:45:33.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dcca59640 1 Processor -- start 2024-03-16T20:45:33.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dcca59640 1 -- start start 2024-03-16T20:45:33.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dcca59640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8db0004e20 0x7f8dc8145ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dcca59640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dc8134c30 0x7f8dc81350b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dcca59640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8dc81403c0 0x7f8dc8140870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dcca59640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc8139c20 con 0x7f8dc8134c30 2024-03-16T20:45:33.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dcca59640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc8134790 con 0x7f8dc81403c0 2024-03-16T20:45:33.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dcca59640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc81ddfe0 con 0x7f8db0004e20 2024-03-16T20:45:33.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dc5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dc8134c30 0x7f8dc81350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dc5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dc8134c30 0x7f8dc81350b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34284/0 (socket says 172.21.15.122:34284) 2024-03-16T20:45:33.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dc5d9b640 1 -- 172.21.15.122:0/1309754212 learned_addr learned my addr 172.21.15.122:0/1309754212 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:33.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dc659c640 1 --2- 172.21.15.122:0/1309754212 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8db0004e20 0x7f8dc8145ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dc6d9d640 1 --2- 172.21.15.122:0/1309754212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8dc81403c0 0x7f8dc8140870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dc5d9b640 1 -- 172.21.15.122:0/1309754212 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8db0004e20 msgr2=0x7f8dc8145ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dc5d9b640 1 --2- 172.21.15.122:0/1309754212 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8db0004e20 0x7f8dc8145ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dc5d9b640 1 -- 172.21.15.122:0/1309754212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8dc81403c0 msgr2=0x7f8dc8140870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dc5d9b640 1 --2- 172.21.15.122:0/1309754212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8dc81403c0 0x7f8dc8140870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dc5d9b640 1 -- 172.21.15.122:0/1309754212 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc81399d0 con 0x7f8dc8134c30 2024-03-16T20:45:33.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dc5d9b640 1 --2- 172.21.15.122:0/1309754212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dc8134c30 0x7f8dc81350b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0115e0 tx=0x7f8dbc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:33.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8db6ffd640 1 -- 172.21.15.122:0/1309754212 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc00f8f0 con 0x7f8dc8134c30 2024-03-16T20:45:33.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8db6ffd640 1 -- 172.21.15.122:0/1309754212 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc00fa50 con 0x7f8dc8134c30 2024-03-16T20:45:33.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8db6ffd640 1 -- 172.21.15.122:0/1309754212 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc00fdc0 con 0x7f8dc8134c30 2024-03-16T20:45:33.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dcca59640 1 -- 172.21.15.122:0/1309754212 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc81ddb50 con 0x7f8dc8134c30 2024-03-16T20:45:33.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.115+0000 7f8dcca59640 1 -- 172.21.15.122:0/1309754212 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc8140db0 con 0x7f8dc8134c30 2024-03-16T20:45:33.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.119+0000 7f8dcca59640 1 -- 172.21.15.122:0/1309754212 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc81ddce0 con 0x7f8dc8134c30 2024-03-16T20:45:33.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.119+0000 7f8db6ffd640 1 -- 172.21.15.122:0/1309754212 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f8dbc015460 con 0x7f8dc8134c30 2024-03-16T20:45:33.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.119+0000 7f8db6ffd640 1 --2- 172.21.15.122:0/1309754212 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8da8066e40 0x7f8da8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.119+0000 7f8db6ffd640 1 -- 172.21.15.122:0/1309754212 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8dbc084f20 con 0x7f8dc8134c30 2024-03-16T20:45:33.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.119+0000 7f8dc659c640 1 --2- 172.21.15.122:0/1309754212 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8da8066e40 0x7f8da8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.119+0000 7f8dc659c640 1 --2- 172.21.15.122:0/1309754212 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8da8066e40 0x7f8da8069300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8139670 tx=0x7f8db800b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:33.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.123+0000 7f8db6ffd640 1 -- 172.21.15.122:0/1309754212 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f8dbc051da0 con 0x7f8dc8134c30 2024-03-16T20:45:33.314 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.311+0000 7f8dcca59640 1 -- 172.21.15.122:0/1309754212 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8dc8075280 con 0x7f8dc8134c30 2024-03-16T20:45:33.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.311+0000 7f8db6ffd640 1 -- 172.21.15.122:0/1309754212 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3397 (secure 0 0 0) 0x7f8dbc022d60 con 0x7f8dc8134c30 2024-03-16T20:45:33.315 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:33.315 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3481092793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3481092793},{"type":"v1","addr":"172.21.15.122:6833","nonce":3481092793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T20:45:04.711887+0000","modified":"2024-03-16T20:45:06.765685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/1428608180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1428608180},{"type":"v1","addr":"172.21.15.195:6835","nonce":1428608180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-03-16T20:45:33.316 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 6 2024-03-16T20:45:33.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.311+0000 7f8dcca59640 1 -- 172.21.15.122:0/1309754212 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8da8066e40 msgr2=0x7f8da8069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.311+0000 7f8dcca59640 1 --2- 172.21.15.122:0/1309754212 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8da8066e40 0x7f8da8069300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8139670 tx=0x7f8db800b040 comp rx=0 tx=0).stop 2024-03-16T20:45:33.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.311+0000 7f8dcca59640 1 -- 172.21.15.122:0/1309754212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dc8134c30 msgr2=0x7f8dc81350b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.311+0000 7f8dcca59640 1 --2- 172.21.15.122:0/1309754212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dc8134c30 0x7f8dc81350b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0115e0 tx=0x7f8dbc011aa0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.311+0000 7f8dcca59640 1 -- 172.21.15.122:0/1309754212 shutdown_connections 2024-03-16T20:45:33.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.311+0000 7f8dcca59640 1 --2- 172.21.15.122:0/1309754212 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f8da8066e40 0x7f8da8069300 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.311+0000 7f8dcca59640 1 --2- 172.21.15.122:0/1309754212 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f8dc81403c0 0x7f8dc8140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.311+0000 7f8dcca59640 1 --2- 172.21.15.122:0/1309754212 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dc8134c30 0x7f8dc81350b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.311+0000 7f8dcca59640 1 --2- 172.21.15.122:0/1309754212 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f8db0004e20 0x7f8dc8145ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.311+0000 7f8dcca59640 1 -- 172.21.15.122:0/1309754212 >> 172.21.15.122:0/1309754212 conn(0x7f8dc80b8860 msgr2=0x7f8dc80b6630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:33.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.311+0000 7f8dcca59640 1 -- 172.21.15.122:0/1309754212 shutdown_connections 2024-03-16T20:45:33.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.311+0000 7f8dcca59640 1 -- 172.21.15.122:0/1309754212 wait complete. 2024-03-16T20:45:33.333 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:33.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.439+0000 7f58c4fdf640 1 Processor -- start 2024-03-16T20:45:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.439+0000 7f58c4fdf640 1 -- start start 2024-03-16T20:45:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.439+0000 7f58c4fdf640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f58c0066fa0 0x7f58c013f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.439+0000 7f58c4fdf640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f58c0072d80 0x7f58c01455a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.439+0000 7f58c4fdf640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c0145110 con 0x7f58c0099c60 2024-03-16T20:45:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.439+0000 7f58c4fdf640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c0067380 con 0x7f58c0072d80 2024-03-16T20:45:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.439+0000 7f58c4fdf640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c009a4f0 con 0x7f58c0066fa0 2024-03-16T20:45:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.439+0000 7f58bdd9b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f58c0072d80 0x7f58c01455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.444 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.439+0000 7f58bdd9b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f58c0072d80 0x7f58c01455a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34290/0 (socket says 172.21.15.122:34290) 2024-03-16T20:45:33.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.439+0000 7f58bdd9b640 1 -- 172.21.15.122:0/3069545527 learned_addr learned my addr 172.21.15.122:0/3069545527 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:33.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.439+0000 7f58be59c640 1 --2- 172.21.15.122:0/3069545527 >> v2:172.21.15.195:3300/0 conn(0x7f58c0066fa0 0x7f58c013f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.439+0000 7f58bdd9b640 1 -- 172.21.15.122:0/3069545527 >> v2:172.21.15.195:3300/0 conn(0x7f58c0066fa0 msgr2=0x7f58c013f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:33.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.439+0000 7f58bdd9b640 1 --2- 172.21.15.122:0/3069545527 >> v2:172.21.15.195:3300/0 conn(0x7f58c0066fa0 0x7f58c013f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.439+0000 7f58bdd9b640 1 -- 172.21.15.122:0/3069545527 >> v1:172.21.15.122:6789/0 conn(0x7f58c0099c60 legacy=0x7f58c01484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.439+0000 7f58bdd9b640 1 -- 172.21.15.122:0/3069545527 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c009a210 con 0x7f58c0072d80 2024-03-16T20:45:33.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.439+0000 7f58bdd9b640 1 --2- 172.21.15.122:0/3069545527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58c0072d80 0x7f58c01455a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f58c009b290 tx=0x7f58b401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8762d530b7abb server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:33.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58bd59a640 1 -- 172.21.15.122:0/3069545527 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40ac600 con 0x7f58c0072d80 2024-03-16T20:45:33.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58bd59a640 1 -- 172.21.15.122:0/3069545527 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b40a8050 con 0x7f58c0072d80 2024-03-16T20:45:33.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58bd59a640 1 -- 172.21.15.122:0/3069545527 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40ac980 con 0x7f58c0072d80 2024-03-16T20:45:33.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 -- 172.21.15.122:0/3069545527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58c0072d80 msgr2=0x7f58c01455a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 --2- 172.21.15.122:0/3069545527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58c0072d80 0x7f58c01455a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f58c009b290 tx=0x7f58b401c490 comp rx=0 tx=0).stop 2024-03-16T20:45:33.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 -- 172.21.15.122:0/3069545527 shutdown_connections 2024-03-16T20:45:33.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 --2- 172.21.15.122:0/3069545527 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58c0072d80 0x7f58c01455a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 --2- 172.21.15.122:0/3069545527 >> v2:172.21.15.195:3300/0 conn(0x7f58c0066fa0 0x7f58c013f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 -- 172.21.15.122:0/3069545527 >> 172.21.15.122:0/3069545527 conn(0x7f58c006d2b0 msgr2=0x7f58c006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:33.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 -- 172.21.15.122:0/3069545527 shutdown_connections 2024-03-16T20:45:33.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 -- 172.21.15.122:0/3069545527 wait complete. 2024-03-16T20:45:33.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 Processor -- start 2024-03-16T20:45:33.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 -- start start 2024-03-16T20:45:33.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58c0066fa0 0x7f58c0138640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58c0072d80 0x7f58c01e1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58c0099c60 0x7f58c01e1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c0137ba0 con 0x7f58c0099c60 2024-03-16T20:45:33.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c0137df0 con 0x7f58c0072d80 2024-03-16T20:45:33.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c013cf70 con 0x7f58c0066fa0 2024-03-16T20:45:33.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58bed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58c0099c60 0x7f58c01e1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58bed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58c0099c60 0x7f58c01e1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34296/0 (socket says 172.21.15.122:34296) 2024-03-16T20:45:33.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58bed9d640 1 -- 172.21.15.122:0/2002247987 learned_addr learned my addr 172.21.15.122:0/2002247987 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:33.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58be59c640 1 --2- 172.21.15.122:0/2002247987 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58c0066fa0 0x7f58c0138640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58bdd9b640 1 --2- 172.21.15.122:0/2002247987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58c0072d80 0x7f58c01e1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58bed9d640 1 -- 172.21.15.122:0/2002247987 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58c0066fa0 msgr2=0x7f58c0138640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58bed9d640 1 --2- 172.21.15.122:0/2002247987 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58c0066fa0 0x7f58c0138640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58bed9d640 1 -- 172.21.15.122:0/2002247987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58c0072d80 msgr2=0x7f58c01e1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58bed9d640 1 --2- 172.21.15.122:0/2002247987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58c0072d80 0x7f58c01e1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58bed9d640 1 -- 172.21.15.122:0/2002247987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b40094d0 con 0x7f58c0099c60 2024-03-16T20:45:33.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58bdd9b640 1 --2- 172.21.15.122:0/2002247987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58c0072d80 0x7f58c01e1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:33.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58bed9d640 1 --2- 172.21.15.122:0/2002247987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58c0099c60 0x7f58c01e1760 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f58b800c760 tx=0x7f58b800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:33.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58aeffd640 1 -- 172.21.15.122:0/2002247987 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b8003b80 con 0x7f58c0099c60 2024-03-16T20:45:33.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58aeffd640 1 -- 172.21.15.122:0/2002247987 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b8003ce0 con 0x7f58c0099c60 2024-03-16T20:45:33.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 -- 172.21.15.122:0/2002247987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c0070a10 con 0x7f58c0099c60 2024-03-16T20:45:33.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58aeffd640 1 -- 172.21.15.122:0/2002247987 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b800aa30 con 0x7f58c0099c60 2024-03-16T20:45:33.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 -- 172.21.15.122:0/2002247987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c0137020 con 0x7f58c0099c60 2024-03-16T20:45:33.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.443+0000 7f58c4fdf640 1 -- 172.21.15.122:0/2002247987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c0075280 con 0x7f58c0099c60 2024-03-16T20:45:33.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.447+0000 7f58aeffd640 1 -- 172.21.15.122:0/2002247987 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f58b800ab90 con 0x7f58c0099c60 2024-03-16T20:45:33.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.447+0000 7f58aeffd640 1 --2- 172.21.15.122:0/2002247987 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f58a0066e40 0x7f58a0069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.447+0000 7f58aeffd640 1 -- 172.21.15.122:0/2002247987 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58b8084800 con 0x7f58c0099c60 2024-03-16T20:45:33.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.447+0000 7f58be59c640 1 --2- 172.21.15.122:0/2002247987 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f58a0066e40 0x7f58a0069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.447+0000 7f58be59c640 1 --2- 172.21.15.122:0/2002247987 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f58a0066e40 0x7f58a0069300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f58b000b9d0 tx=0x7f58b0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:33.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.451+0000 7f58aeffd640 1 -- 172.21.15.122:0/2002247987 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f58b80515f0 con 0x7f58c0099c60 2024-03-16T20:45:33.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.599+0000 7f58c4fdf640 1 -- 172.21.15.122:0/2002247987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58c009bd90 con 0x7f58c0099c60 2024-03-16T20:45:33.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.599+0000 7f58aeffd640 1 -- 172.21.15.122:0/2002247987 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7f58b8054cb0 con 0x7f58c0099c60 2024-03-16T20:45:33.604 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:33.604 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:06.404479+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:33.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.603+0000 7f58c4fdf640 1 -- 172.21.15.122:0/2002247987 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f58a0066e40 msgr2=0x7f58a0069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.603+0000 7f58c4fdf640 1 --2- 172.21.15.122:0/2002247987 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f58a0066e40 0x7f58a0069300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f58b000b9d0 tx=0x7f58b0008040 comp rx=0 tx=0).stop 2024-03-16T20:45:33.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.603+0000 7f58c4fdf640 1 -- 172.21.15.122:0/2002247987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58c0099c60 msgr2=0x7f58c01e1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.603+0000 7f58c4fdf640 1 --2- 172.21.15.122:0/2002247987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58c0099c60 0x7f58c01e1760 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f58b800c760 tx=0x7f58b800cc20 comp rx=0 tx=0).stop 2024-03-16T20:45:33.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.603+0000 7f58c4fdf640 1 -- 172.21.15.122:0/2002247987 shutdown_connections 2024-03-16T20:45:33.606 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.603+0000 7f58c4fdf640 1 --2- 172.21.15.122:0/2002247987 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f58a0066e40 0x7f58a0069300 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.603+0000 7f58c4fdf640 1 --2- 172.21.15.122:0/2002247987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f58c0099c60 0x7f58c01e1760 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.603+0000 7f58c4fdf640 1 --2- 172.21.15.122:0/2002247987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f58c0072d80 0x7f58c01e1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.603+0000 7f58c4fdf640 1 --2- 172.21.15.122:0/2002247987 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f58c0066fa0 0x7f58c0138640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.603+0000 7f58c4fdf640 1 -- 172.21.15.122:0/2002247987 >> 172.21.15.122:0/2002247987 conn(0x7f58c006d2b0 msgr2=0x7f58c0143930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:33.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.603+0000 7f58c4fdf640 1 -- 172.21.15.122:0/2002247987 shutdown_connections 2024-03-16T20:45:33.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.603+0000 7f58c4fdf640 1 -- 172.21.15.122:0/2002247987 wait complete. 2024-03-16T20:45:33.621 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-16T20:45:33.622 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T20:45:33.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.727+0000 7f095e90e640 1 Processor -- start 2024-03-16T20:45:33.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.727+0000 7f095e90e640 1 -- start start 2024-03-16T20:45:33.733 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.727+0000 7f095e90e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0958146a50 0x7f0958143460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.727+0000 7f095e90e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09580b4c50 con 0x7f095813d8d0 2024-03-16T20:45:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.727+0000 7f095e90e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0958146ee0 con 0x7f0958071580 2024-03-16T20:45:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.727+0000 7f095e90e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09580b2c30 con 0x7f0958146a50 2024-03-16T20:45:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.727+0000 7f09577fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0958146a50 0x7f0958143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.727+0000 7f09577fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f0958146a50 0x7f0958143460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34306/0 (socket says 172.21.15.122:34306) 2024-03-16T20:45:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.727+0000 7f09577fe640 1 -- 172.21.15.122:0/931731991 learned_addr learned my addr 172.21.15.122:0/931731991 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.727+0000 7f09577fe640 1 -- 172.21.15.122:0/931731991 >> v1:172.21.15.195:6789/0 conn(0x7f0958071580 legacy=0x7f095806af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.727+0000 7f09577fe640 1 -- 172.21.15.122:0/931731991 >> v1:172.21.15.122:6789/0 conn(0x7f095813d8d0 legacy=0x7f095806b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.727+0000 7f09577fe640 1 -- 172.21.15.122:0/931731991 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09581471c0 con 0x7f0958146a50 2024-03-16T20:45:33.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.727+0000 7f0956ffd640 1 -- 172.21.15.122:0/931731991 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719506714 0 0) 0x7f09580b4c50 con 0x7f095813d8d0 2024-03-16T20:45:33.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.731+0000 7f09577fe640 1 --2- 172.21.15.122:0/931731991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0958146a50 0x7f0958143460 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f094c009870 tx=0x7f094c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738bcd4170739833 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:33.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.731+0000 7f0956ffd640 1 -- 172.21.15.122:0/931731991 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c0a7070 con 0x7f0958146a50 2024-03-16T20:45:33.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.731+0000 7f0956ffd640 1 -- 172.21.15.122:0/931731991 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c0ab430 con 0x7f0958146a50 2024-03-16T20:45:33.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.731+0000 7f0956ffd640 1 -- 172.21.15.122:0/931731991 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c0ab7c0 con 0x7f0958146a50 2024-03-16T20:45:33.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.731+0000 7f095e90e640 1 -- 172.21.15.122:0/931731991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0958146a50 msgr2=0x7f0958143460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.731+0000 7f095e90e640 1 --2- 172.21.15.122:0/931731991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0958146a50 0x7f0958143460 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f094c009870 tx=0x7f094c01c520 comp rx=0 tx=0).stop 2024-03-16T20:45:33.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.731+0000 7f095e90e640 1 -- 172.21.15.122:0/931731991 shutdown_connections 2024-03-16T20:45:33.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.731+0000 7f095e90e640 1 --2- 172.21.15.122:0/931731991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0958146a50 0x7f0958143460 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.731+0000 7f095e90e640 1 -- 172.21.15.122:0/931731991 >> 172.21.15.122:0/931731991 conn(0x7f09580b8860 msgr2=0x7f09580b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:33.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.731+0000 7f095e90e640 1 -- 172.21.15.122:0/931731991 shutdown_connections 2024-03-16T20:45:33.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.731+0000 7f095e90e640 1 -- 172.21.15.122:0/931731991 wait complete. 2024-03-16T20:45:33.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.731+0000 7f095e90e640 1 Processor -- start 2024-03-16T20:45:33.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.731+0000 7f095e90e640 1 -- start start 2024-03-16T20:45:33.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.731+0000 7f095e90e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0958071580 0x7f09580b7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f095e90e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095813d8d0 0x7f09581dd820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f095e90e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09581ddd60 0x7f09581e0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f095e90e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0958144900 con 0x7f095813d8d0 2024-03-16T20:45:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f095e90e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0958144620 con 0x7f0958071580 2024-03-16T20:45:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f095e90e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0958145bb0 con 0x7f09581ddd60 2024-03-16T20:45:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f09577fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095813d8d0 0x7f09581dd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f09577fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095813d8d0 0x7f09581dd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34308/0 (socket says 172.21.15.122:34308) 2024-03-16T20:45:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f09577fe640 1 -- 172.21.15.122:0/3753294953 learned_addr learned my addr 172.21.15.122:0/3753294953 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f09577fe640 1 -- 172.21.15.122:0/3753294953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09581ddd60 msgr2=0x7f09581e0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f0957fff640 1 --2- 172.21.15.122:0/3753294953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0958071580 0x7f09580b7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f09577fe640 1 --2- 172.21.15.122:0/3753294953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09581ddd60 0x7f09581e0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f09577fe640 1 -- 172.21.15.122:0/3753294953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0958071580 msgr2=0x7f09580b7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f09577fe640 1 --2- 172.21.15.122:0/3753294953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0958071580 0x7f09580b7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f09577fe640 1 -- 172.21.15.122:0/3753294953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c0094d0 con 0x7f095813d8d0 2024-03-16T20:45:33.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f09577fe640 1 --2- 172.21.15.122:0/3753294953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095813d8d0 0x7f09581dd820 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f094c0ad040 tx=0x7f094c01ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f0954ff9640 1 -- 172.21.15.122:0/3753294953 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c0abdd0 con 0x7f095813d8d0 2024-03-16T20:45:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f0954ff9640 1 -- 172.21.15.122:0/3753294953 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c0b8070 con 0x7f095813d8d0 2024-03-16T20:45:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f095e90e640 1 -- 172.21.15.122:0/3753294953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0958143db0 con 0x7f095813d8d0 2024-03-16T20:45:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f0954ff9640 1 -- 172.21.15.122:0/3753294953 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c004240 con 0x7f095813d8d0 2024-03-16T20:45:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f095e90e640 1 -- 172.21.15.122:0/3753294953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0958146690 con 0x7f095813d8d0 2024-03-16T20:45:33.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f095e90e640 1 -- 172.21.15.122:0/3753294953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0958144ab0 con 0x7f095813d8d0 2024-03-16T20:45:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f0954ff9640 1 -- 172.21.15.122:0/3753294953 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f094c0043a0 con 0x7f095813d8d0 2024-03-16T20:45:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f0954ff9640 1 --2- 172.21.15.122:0/3753294953 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0944066e40 0x7f0944069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f0954ff9640 1 -- 172.21.15.122:0/3753294953 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f094c11ed00 con 0x7f095813d8d0 2024-03-16T20:45:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f0957fff640 1 --2- 172.21.15.122:0/3753294953 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0944066e40 0x7f0944069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:33.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.735+0000 7f0957fff640 1 --2- 172.21.15.122:0/3753294953 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0944066e40 0x7f0944069300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f09480012f0 tx=0x7f0948009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:33.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.739+0000 7f0954ff9640 1 -- 172.21.15.122:0/3753294953 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f094c0ebd80 con 0x7f095813d8d0 2024-03-16T20:45:33.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.915+0000 7f095e90e640 1 -- 172.21.15.122:0/3753294953 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0958144ab0 con 0x7f095813d8d0 2024-03-16T20:45:33.922 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.919+0000 7f0954ff9640 1 -- 172.21.15.122:0/3753294953 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3397 (secure 0 0 0) 0x7f0958144ab0 con 0x7f095813d8d0 2024-03-16T20:45:33.923 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:33.923 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3481092793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3481092793},{"type":"v1","addr":"172.21.15.122:6833","nonce":3481092793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T20:45:04.711887+0000","modified":"2024-03-16T20:45:06.765685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/1428608180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1428608180},{"type":"v1","addr":"172.21.15.195:6835","nonce":1428608180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-03-16T20:45:33.923 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 6 2024-03-16T20:45:33.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.919+0000 7f095e90e640 1 -- 172.21.15.122:0/3753294953 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0944066e40 msgr2=0x7f0944069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.919+0000 7f095e90e640 1 --2- 172.21.15.122:0/3753294953 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0944066e40 0x7f0944069300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f09480012f0 tx=0x7f0948009040 comp rx=0 tx=0).stop 2024-03-16T20:45:33.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.919+0000 7f095e90e640 1 -- 172.21.15.122:0/3753294953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095813d8d0 msgr2=0x7f09581dd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:33.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.919+0000 7f095e90e640 1 --2- 172.21.15.122:0/3753294953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095813d8d0 0x7f09581dd820 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f094c0ad040 tx=0x7f094c01ad60 comp rx=0 tx=0).stop 2024-03-16T20:45:33.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.919+0000 7f095e90e640 1 -- 172.21.15.122:0/3753294953 shutdown_connections 2024-03-16T20:45:33.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.919+0000 7f095e90e640 1 --2- 172.21.15.122:0/3753294953 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0944066e40 0x7f0944069300 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.919+0000 7f095e90e640 1 --2- 172.21.15.122:0/3753294953 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f09581ddd60 0x7f09581e0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.919+0000 7f095e90e640 1 --2- 172.21.15.122:0/3753294953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f095813d8d0 0x7f09581dd820 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.919+0000 7f095e90e640 1 --2- 172.21.15.122:0/3753294953 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0958071580 0x7f09580b7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:33.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.919+0000 7f095e90e640 1 -- 172.21.15.122:0/3753294953 >> 172.21.15.122:0/3753294953 conn(0x7f09580b8860 msgr2=0x7f09580b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:33.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.919+0000 7f095e90e640 1 -- 172.21.15.122:0/3753294953 shutdown_connections 2024-03-16T20:45:33.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:33.919+0000 7f095e90e640 1 -- 172.21.15.122:0/3753294953 wait complete. 2024-03-16T20:45:33.940 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T20:45:34.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.039+0000 7fb95b294640 1 Processor -- start 2024-03-16T20:45:34.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.039+0000 7fb95b294640 1 -- start start 2024-03-16T20:45:34.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb95b294640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb954148b40 0x7fb95413f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb95b294640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9540b02a0 con 0x7fb954072120 2024-03-16T20:45:34.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb95b294640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9540b1050 con 0x7fb95413ed60 2024-03-16T20:45:34.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb95b294640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb954072500 con 0x7fb954148b40 2024-03-16T20:45:34.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb959030640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fb954072120 0x7fb9540afb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39864/0 (socket says 172.21.15.122:39864) 2024-03-16T20:45:34.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb959030640 1 -- 172.21.15.122:0/3775894685 learned_addr learned my addr 172.21.15.122:0/3775894685 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:34.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 -- 172.21.15.122:0/3775894685 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664052123 0 0) 0x7fb9540b02a0 con 0x7fb954072120 2024-03-16T20:45:34.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 -- 172.21.15.122:0/3775894685 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb93c003530 con 0x7fb954072120 2024-03-16T20:45:34.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb959831640 1 --2- 172.21.15.122:0/3775894685 >> v2:172.21.15.195:3300/0 conn(0x7fb954148b40 0x7fb95413f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 -- 172.21.15.122:0/3775894685 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 730039342 0 0) 0x7fb93c003530 con 0x7fb954072120 2024-03-16T20:45:34.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 -- 172.21.15.122:0/3775894685 >> v2:172.21.15.195:3300/0 conn(0x7fb954148b40 msgr2=0x7fb95413f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:34.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 --2- 172.21.15.122:0/3775894685 >> v2:172.21.15.195:3300/0 conn(0x7fb954148b40 0x7fb95413f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 -- 172.21.15.122:0/3775894685 >> v1:172.21.15.195:6789/0 conn(0x7fb95413ed60 legacy=0x7fb954148430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 -- 172.21.15.122:0/3775894685 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9540b38a0 con 0x7fb954072120 2024-03-16T20:45:34.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 -- 172.21.15.122:0/3775894685 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7fb9440016a0 con 0x7fb954072120 2024-03-16T20:45:34.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 -- 172.21.15.122:0/3775894685 >> v1:172.21.15.122:6789/0 conn(0x7fb954072120 legacy=0x7fb9540afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 --2- 172.21.15.122:0/3775894685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb93c003980 0x7fb93c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 --2- 172.21.15.122:0/3775894685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb93c0047d0 0x7fb93c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 --2- 172.21.15.122:0/3775894685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb93c0052b0 0x7fb93c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 -- 172.21.15.122:0/3775894685 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb954072500 con 0x7fb93c0047d0 2024-03-16T20:45:34.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 -- 172.21.15.122:0/3775894685 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c00c180 con 0x7fb93c003980 2024-03-16T20:45:34.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 -- 172.21.15.122:0/3775894685 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c00c340 con 0x7fb93c0052b0 2024-03-16T20:45:34.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb959831640 1 --2- 172.21.15.122:0/3775894685 >> v2:172.21.15.195:3300/0 conn(0x7fb954148b40 0x7fb95413f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:34.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb95882f640 1 --2- 172.21.15.122:0/3775894685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb93c0047d0 0x7fb93c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb95882f640 1 -- 172.21.15.122:0/3775894685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb93c0052b0 msgr2=0x7fb93c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:34.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb95882f640 1 --2- 172.21.15.122:0/3775894685 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb93c0052b0 0x7fb93c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb95882f640 1 -- 172.21.15.122:0/3775894685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb93c003980 msgr2=0x7fb93c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:34.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb959030640 1 --2- 172.21.15.122:0/3775894685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb93c003980 0x7fb93c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb95882f640 1 --2- 172.21.15.122:0/3775894685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb93c003980 0x7fb93c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb95882f640 1 -- 172.21.15.122:0/3775894685 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb93c003530 con 0x7fb93c0047d0 2024-03-16T20:45:34.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb959831640 1 -- 172.21.15.122:0/3775894685 reap_dead start 2024-03-16T20:45:34.050 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb959030640 1 --2- 172.21.15.122:0/3775894685 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb93c003980 0x7fb93c003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:34.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb95882f640 1 --2- 172.21.15.122:0/3775894685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb93c0047d0 0x7fb93c004c50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb948001750 tx=0x7fb94801c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6c80b0693a50ef server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:34.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 -- 172.21.15.122:0/3775894685 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9480052d0 con 0x7fb93c0047d0 2024-03-16T20:45:34.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 -- 172.21.15.122:0/3775894685 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9480a7050 con 0x7fb93c0047d0 2024-03-16T20:45:34.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.043+0000 7fb953fff640 1 -- 172.21.15.122:0/3775894685 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb948003600 con 0x7fb93c0047d0 2024-03-16T20:45:34.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95b294640 1 -- 172.21.15.122:0/3775894685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb93c0047d0 msgr2=0x7fb93c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95b294640 1 --2- 172.21.15.122:0/3775894685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb93c0047d0 0x7fb93c004c50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb948001750 tx=0x7fb94801c310 comp rx=0 tx=0).stop 2024-03-16T20:45:34.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95b294640 1 -- 172.21.15.122:0/3775894685 shutdown_connections 2024-03-16T20:45:34.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95b294640 1 --2- 172.21.15.122:0/3775894685 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb93c0047d0 0x7fb93c004c50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95b294640 1 -- 172.21.15.122:0/3775894685 >> 172.21.15.122:0/3775894685 conn(0x7fb9540b8860 msgr2=0x7fb9540b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:34.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95b294640 1 -- 172.21.15.122:0/3775894685 shutdown_connections 2024-03-16T20:45:34.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95b294640 1 -- 172.21.15.122:0/3775894685 wait complete. 2024-03-16T20:45:34.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95b294640 1 Processor -- start 2024-03-16T20:45:34.053 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95b294640 1 -- start start 2024-03-16T20:45:34.055 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95b294640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb93c0047d0 0x7fb95413dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95b294640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9541d0f10 0x7fb9541ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95b294640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9541de1d0 0x7fb954139750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95b294640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9541416c0 con 0x7fb9541de1d0 2024-03-16T20:45:34.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95b294640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9541427f0 con 0x7fb9541d0f10 2024-03-16T20:45:34.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95b294640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9541d16d0 con 0x7fb93c0047d0 2024-03-16T20:45:34.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb959831640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9541de1d0 0x7fb954139750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb959831640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9541de1d0 0x7fb954139750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34330/0 (socket says 172.21.15.122:34330) 2024-03-16T20:45:34.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb959831640 1 -- 172.21.15.122:0/1551084398 learned_addr learned my addr 172.21.15.122:0/1551084398 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:34.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb959030640 1 --2- 172.21.15.122:0/1551084398 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb93c0047d0 0x7fb95413dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95882f640 1 --2- 172.21.15.122:0/1551084398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9541d0f10 0x7fb9541ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb959831640 1 -- 172.21.15.122:0/1551084398 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb93c0047d0 msgr2=0x7fb95413dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.056 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb959831640 1 --2- 172.21.15.122:0/1551084398 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb93c0047d0 0x7fb95413dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb959831640 1 -- 172.21.15.122:0/1551084398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9541d0f10 msgr2=0x7fb9541ddc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb959831640 1 --2- 172.21.15.122:0/1551084398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9541d0f10 0x7fb9541ddc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb959831640 1 -- 172.21.15.122:0/1551084398 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb954142510 con 0x7fb9541de1d0 2024-03-16T20:45:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb959831640 1 --2- 172.21.15.122:0/1551084398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9541de1d0 0x7fb954139750 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb9540b05f0 tx=0x7fb94c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb951ffb640 1 -- 172.21.15.122:0/1551084398 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c007e90 con 0x7fb9541de1d0 2024-03-16T20:45:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb951ffb640 1 -- 172.21.15.122:0/1551084398 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c00f040 con 0x7fb9541de1d0 2024-03-16T20:45:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95b294640 1 -- 172.21.15.122:0/1551084398 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9541413e0 con 0x7fb9541de1d0 2024-03-16T20:45:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb951ffb640 1 -- 172.21.15.122:0/1551084398 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c015640 con 0x7fb9541de1d0 2024-03-16T20:45:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.047+0000 7fb95b294640 1 -- 172.21.15.122:0/1551084398 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9540af770 con 0x7fb9541de1d0 2024-03-16T20:45:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.051+0000 7fb95b294640 1 -- 172.21.15.122:0/1551084398 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9541d0500 con 0x7fb9541de1d0 2024-03-16T20:45:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.051+0000 7fb951ffb640 1 -- 172.21.15.122:0/1551084398 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fb94c01e3b0 con 0x7fb9541de1d0 2024-03-16T20:45:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.051+0000 7fb951ffb640 1 --2- 172.21.15.122:0/1551084398 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb940066e40 0x7fb940069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.057 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.051+0000 7fb951ffb640 1 -- 172.21.15.122:0/1551084398 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb94c084a30 con 0x7fb9541de1d0 2024-03-16T20:45:34.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.051+0000 7fb959030640 1 --2- 172.21.15.122:0/1551084398 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb940066e40 0x7fb940069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.058 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.051+0000 7fb959030640 1 --2- 172.21.15.122:0/1551084398 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb940066e40 0x7fb940069300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb944001ec0 tx=0x7fb944009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:34.060 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.055+0000 7fb951ffb640 1 -- 172.21.15.122:0/1551084398 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb9541d0500 con 0x7fb9541de1d0 2024-03-16T20:45:34.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.235+0000 7fb95b294640 1 -- 172.21.15.122:0/1551084398 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9541d0500 con 0x7fb9541de1d0 2024-03-16T20:45:34.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.235+0000 7fb951ffb640 1 -- 172.21.15.122:0/1551084398 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+3397 (secure 0 0 0) 0x7fb94c01ad60 con 0x7fb9541de1d0 2024-03-16T20:45:34.240 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:34.240 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4242,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/3481092793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":3481092793},{"type":"v1","addr":"172.21.15.122:6833","nonce":3481092793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T20:45:04.711887+0000","modified":"2024-03-16T20:45:06.765685+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4251":{"gid":4251,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.195:6835/1428608180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":1428608180},{"type":"v1","addr":"172.21.15.195:6835","nonce":1428608180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4251,"qdb_cluster":[4251]},"id":1}]} 2024-03-16T20:45:34.240 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 6 2024-03-16T20:45:34.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.235+0000 7fb95b294640 1 -- 172.21.15.122:0/1551084398 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb940066e40 msgr2=0x7fb940069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.235+0000 7fb95b294640 1 --2- 172.21.15.122:0/1551084398 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb940066e40 0x7fb940069300 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb944001ec0 tx=0x7fb944009040 comp rx=0 tx=0).stop 2024-03-16T20:45:34.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.235+0000 7fb95b294640 1 -- 172.21.15.122:0/1551084398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9541de1d0 msgr2=0x7fb954139750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.235+0000 7fb95b294640 1 --2- 172.21.15.122:0/1551084398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9541de1d0 0x7fb954139750 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb9540b05f0 tx=0x7fb94c00d4a0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.235+0000 7fb95b294640 1 -- 172.21.15.122:0/1551084398 shutdown_connections 2024-03-16T20:45:34.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.235+0000 7fb95b294640 1 --2- 172.21.15.122:0/1551084398 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb940066e40 0x7fb940069300 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.235+0000 7fb95b294640 1 --2- 172.21.15.122:0/1551084398 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9541de1d0 0x7fb954139750 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.235+0000 7fb95b294640 1 --2- 172.21.15.122:0/1551084398 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb9541d0f10 0x7fb9541ddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.235+0000 7fb95b294640 1 --2- 172.21.15.122:0/1551084398 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb93c0047d0 0x7fb95413dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.239+0000 7fb95b294640 1 -- 172.21.15.122:0/1551084398 >> 172.21.15.122:0/1551084398 conn(0x7fb9540b8860 msgr2=0x7fb954146bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:34.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.239+0000 7fb95b294640 1 -- 172.21.15.122:0/1551084398 shutdown_connections 2024-03-16T20:45:34.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.239+0000 7fb95b294640 1 -- 172.21.15.122:0/1551084398 wait complete. 2024-03-16T20:45:34.257 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:34.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.359+0000 7fe935ed2640 1 Processor -- start 2024-03-16T20:45:34.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.359+0000 7fe935ed2640 1 -- start start 2024-03-16T20:45:34.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.359+0000 7fe935ed2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe93013ed60 0x7fe93013f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.359+0000 7fe935ed2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe930148430 0x7fe9301488c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.359+0000 7fe935ed2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9300b02a0 con 0x7fe930072120 2024-03-16T20:45:34.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.359+0000 7fe935ed2640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9300b1050 con 0x7fe93013ed60 2024-03-16T20:45:34.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.359+0000 7fe935ed2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe930072500 con 0x7fe930148430 2024-03-16T20:45:34.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.363+0000 7fe92ffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe930148430 0x7fe9301488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.363+0000 7fe92effd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe93013ed60 0x7fe93013f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.363+0000 7fe92f7fe640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fe930072120 0x7fe9300afb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.122:57722/0 (socket says 172.21.15.122:57722) 2024-03-16T20:45:34.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.363+0000 7fe92f7fe640 1 -- 172.21.15.122:0/135025044 learned_addr learned my addr 172.21.15.122:0/135025044 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:34.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.363+0000 7fe92ffff640 1 -- 172.21.15.122:0/135025044 >> v2:172.21.15.195:3300/0 conn(0x7fe93013ed60 msgr2=0x7fe93013f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:34.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.363+0000 7fe92ffff640 1 --2- 172.21.15.122:0/135025044 >> v2:172.21.15.195:3300/0 conn(0x7fe93013ed60 0x7fe93013f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.363+0000 7fe92ffff640 1 -- 172.21.15.122:0/135025044 >> v1:172.21.15.195:6789/0 conn(0x7fe930072120 legacy=0x7fe9300afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.367 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.363+0000 7fe92ffff640 1 -- 172.21.15.122:0/135025044 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9300b38a0 con 0x7fe930148430 2024-03-16T20:45:34.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.367+0000 7fe92ffff640 1 --2- 172.21.15.122:0/135025044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe930148430 0x7fe9301488c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe9200097c0 tx=0x7fe92001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7819f68c00acabe8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:34.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.367+0000 7fe92e7fc640 1 -- 172.21.15.122:0/135025044 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9200a4ea0 con 0x7fe930148430 2024-03-16T20:45:34.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.367+0000 7fe92e7fc640 1 -- 172.21.15.122:0/135025044 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9200a7050 con 0x7fe930148430 2024-03-16T20:45:34.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.367+0000 7fe92e7fc640 1 -- 172.21.15.122:0/135025044 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9200ab820 con 0x7fe930148430 2024-03-16T20:45:34.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.367+0000 7fe935ed2640 1 -- 172.21.15.122:0/135025044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe930148430 msgr2=0x7fe9301488c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.367+0000 7fe935ed2640 1 --2- 172.21.15.122:0/135025044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe930148430 0x7fe9301488c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe9200097c0 tx=0x7fe92001c410 comp rx=0 tx=0).stop 2024-03-16T20:45:34.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.367+0000 7fe935ed2640 1 -- 172.21.15.122:0/135025044 shutdown_connections 2024-03-16T20:45:34.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.367+0000 7fe935ed2640 1 --2- 172.21.15.122:0/135025044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe930148430 0x7fe9301488c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.367+0000 7fe935ed2640 1 --2- 172.21.15.122:0/135025044 >> v2:172.21.15.195:3300/0 conn(0x7fe93013ed60 0x7fe93013f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.367+0000 7fe935ed2640 1 -- 172.21.15.122:0/135025044 >> 172.21.15.122:0/135025044 conn(0x7fe9300b8860 msgr2=0x7fe9300b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:34.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.367+0000 7fe935ed2640 1 -- 172.21.15.122:0/135025044 shutdown_connections 2024-03-16T20:45:34.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.367+0000 7fe935ed2640 1 -- 172.21.15.122:0/135025044 wait complete. 2024-03-16T20:45:34.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.367+0000 7fe935ed2640 1 Processor -- start 2024-03-16T20:45:34.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.367+0000 7fe935ed2640 1 -- start start 2024-03-16T20:45:34.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe935ed2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe930072120 0x7fe930141920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe935ed2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe93013ed60 0x7fe93014ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe935ed2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe930148430 0x7fe93014f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe935ed2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9301e0ed0 con 0x7fe930148430 2024-03-16T20:45:34.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe935ed2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe930142990 con 0x7fe930072120 2024-03-16T20:45:34.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe935ed2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe930142be0 con 0x7fe93013ed60 2024-03-16T20:45:34.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe92ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe930148430 0x7fe93014f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe92ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe930148430 0x7fe93014f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34344/0 (socket says 172.21.15.122:34344) 2024-03-16T20:45:34.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe92ffff640 1 -- 172.21.15.122:0/460839849 learned_addr learned my addr 172.21.15.122:0/460839849 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:34.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe92f7fe640 1 --2- 172.21.15.122:0/460839849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe930072120 0x7fe930141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe92effd640 1 --2- 172.21.15.122:0/460839849 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe93013ed60 0x7fe93014ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe92ffff640 1 -- 172.21.15.122:0/460839849 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe93013ed60 msgr2=0x7fe93014ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe92ffff640 1 --2- 172.21.15.122:0/460839849 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe93013ed60 0x7fe93014ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe92ffff640 1 -- 172.21.15.122:0/460839849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe930072120 msgr2=0x7fe930141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe92ffff640 1 --2- 172.21.15.122:0/460839849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe930072120 0x7fe930141920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe92ffff640 1 -- 172.21.15.122:0/460839849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe920009420 con 0x7fe930148430 2024-03-16T20:45:34.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe92f7fe640 1 --2- 172.21.15.122:0/460839849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe930072120 0x7fe930141920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:34.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe92ffff640 1 --2- 172.21.15.122:0/460839849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe930148430 0x7fe93014f180 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe9200097c0 tx=0x7fe9200191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:34.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe927fff640 1 -- 172.21.15.122:0/460839849 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9200a4ea0 con 0x7fe930148430 2024-03-16T20:45:34.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe927fff640 1 -- 172.21.15.122:0/460839849 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe920006e20 con 0x7fe930148430 2024-03-16T20:45:34.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe935ed2640 1 -- 172.21.15.122:0/460839849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9301496d0 con 0x7fe930148430 2024-03-16T20:45:34.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe927fff640 1 -- 172.21.15.122:0/460839849 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9200b4620 con 0x7fe930148430 2024-03-16T20:45:34.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe935ed2640 1 -- 172.21.15.122:0/460839849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9301e13a0 con 0x7fe930148430 2024-03-16T20:45:34.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe935ed2640 1 -- 172.21.15.122:0/460839849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe930075280 con 0x7fe930148430 2024-03-16T20:45:34.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe927fff640 1 -- 172.21.15.122:0/460839849 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe9200b2040 con 0x7fe930148430 2024-03-16T20:45:34.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe927fff640 1 --2- 172.21.15.122:0/460839849 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe910066e70 0x7fe910069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe927fff640 1 -- 172.21.15.122:0/460839849 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe92011e6e0 con 0x7fe930148430 2024-03-16T20:45:34.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe92f7fe640 1 --2- 172.21.15.122:0/460839849 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe910066e70 0x7fe910069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.371+0000 7fe92f7fe640 1 --2- 172.21.15.122:0/460839849 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe910066e70 0x7fe910069330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe9301480b0 tx=0x7fe928009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:34.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.375+0000 7fe927fff640 1 -- 172.21.15.122:0/460839849 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe9200eb560 con 0x7fe930148430 2024-03-16T20:45:34.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.523+0000 7fe935ed2640 1 -- 172.21.15.122:0/460839849 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9301493f0 con 0x7fe930148430 2024-03-16T20:45:34.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.523+0000 7fe927fff640 1 -- 172.21.15.122:0/460839849 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16641 (secure 0 0 0) 0x7fe9200eec20 con 0x7fe930148430 2024-03-16T20:45:34.529 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:34.529 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":21,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:06.404479+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:02.758515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:03.710783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:34.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.527+0000 7fe935ed2640 1 -- 172.21.15.122:0/460839849 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe910066e70 msgr2=0x7fe910069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.527+0000 7fe935ed2640 1 --2- 172.21.15.122:0/460839849 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe910066e70 0x7fe910069330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe9301480b0 tx=0x7fe928009040 comp rx=0 tx=0).stop 2024-03-16T20:45:34.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.527+0000 7fe935ed2640 1 -- 172.21.15.122:0/460839849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe930148430 msgr2=0x7fe93014f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.527+0000 7fe935ed2640 1 --2- 172.21.15.122:0/460839849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe930148430 0x7fe93014f180 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe9200097c0 tx=0x7fe9200191b0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.527+0000 7fe935ed2640 1 -- 172.21.15.122:0/460839849 shutdown_connections 2024-03-16T20:45:34.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.527+0000 7fe935ed2640 1 --2- 172.21.15.122:0/460839849 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe910066e70 0x7fe910069330 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.527+0000 7fe935ed2640 1 --2- 172.21.15.122:0/460839849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe930148430 0x7fe93014f180 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.527+0000 7fe935ed2640 1 --2- 172.21.15.122:0/460839849 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe93013ed60 0x7fe93014ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.527+0000 7fe935ed2640 1 --2- 172.21.15.122:0/460839849 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe930072120 0x7fe930141920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.527+0000 7fe935ed2640 1 -- 172.21.15.122:0/460839849 >> 172.21.15.122:0/460839849 conn(0x7fe9300b8860 msgr2=0x7fe9300b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:34.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.527+0000 7fe935ed2640 1 -- 172.21.15.122:0/460839849 shutdown_connections 2024-03-16T20:45:34.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.527+0000 7fe935ed2640 1 -- 172.21.15.122:0/460839849 wait complete. 2024-03-16T20:45:34.546 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-16T20:45:34.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.651+0000 7f54f6de9640 1 Processor -- start 2024-03-16T20:45:34.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.651+0000 7f54f6de9640 1 -- start start 2024-03-16T20:45:34.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.651+0000 7f54f6de9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f54f0141580 0x7f54f00baac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.651+0000 7f54f6de9640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f54f006d350 0x7f54f0151920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.651+0000 7f54f6de9640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54f0147ff0 con 0x7f54f00bb000 2024-03-16T20:45:34.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.651+0000 7f54f6de9640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f00bb460 con 0x7f54f006d350 2024-03-16T20:45:34.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.651+0000 7f54f6de9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54f01479a0 con 0x7f54f0141580 2024-03-16T20:45:34.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.651+0000 7f54effff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f54f00bb000 0x7f54f006cc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39876/0 (socket says 172.21.15.122:39876) 2024-03-16T20:45:34.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.651+0000 7f54effff640 1 -- 172.21.15.122:0/3503520806 learned_addr learned my addr 172.21.15.122:0/3503520806 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:34.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.651+0000 7f54f5386640 1 --2- 172.21.15.122:0/3503520806 >> v2:172.21.15.122:3300/0 conn(0x7f54f006d350 0x7f54f0151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.651+0000 7f54f4b85640 1 --2- 172.21.15.122:0/3503520806 >> v2:172.21.15.195:3300/0 conn(0x7f54f0141580 0x7f54f00baac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.651+0000 7f54f5386640 1 -- 172.21.15.122:0/3503520806 >> v2:172.21.15.195:3300/0 conn(0x7f54f0141580 msgr2=0x7f54f00baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:34.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.651+0000 7f54f5386640 1 --2- 172.21.15.122:0/3503520806 >> v2:172.21.15.195:3300/0 conn(0x7f54f0141580 0x7f54f00baac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.651+0000 7f54f5386640 1 -- 172.21.15.122:0/3503520806 >> v1:172.21.15.122:6789/0 conn(0x7f54f00bb000 legacy=0x7f54f006cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.651+0000 7f54ef7fe640 1 -- 172.21.15.122:0/3503520806 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563225212 0 0) 0x7f54f0147ff0 con 0x7f54f00bb000 2024-03-16T20:45:34.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.651+0000 7f54f5386640 1 -- 172.21.15.122:0/3503520806 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f0147c80 con 0x7f54f006d350 2024-03-16T20:45:34.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f5386640 1 --2- 172.21.15.122:0/3503520806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f006d350 0x7f54f0151920 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f54f0067c40 tx=0x7f54e801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d15d96fb6a0e2d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:34.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54ef7fe640 1 -- 172.21.15.122:0/3503520806 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e80a5900 con 0x7f54f006d350 2024-03-16T20:45:34.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54ef7fe640 1 -- 172.21.15.122:0/3503520806 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e80a5a60 con 0x7f54f006d350 2024-03-16T20:45:34.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54ef7fe640 1 -- 172.21.15.122:0/3503520806 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e80a5df0 con 0x7f54f006d350 2024-03-16T20:45:34.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 -- 172.21.15.122:0/3503520806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f006d350 msgr2=0x7f54f0151920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 --2- 172.21.15.122:0/3503520806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f006d350 0x7f54f0151920 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f54f0067c40 tx=0x7f54e801c930 comp rx=0 tx=0).stop 2024-03-16T20:45:34.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 -- 172.21.15.122:0/3503520806 shutdown_connections 2024-03-16T20:45:34.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 --2- 172.21.15.122:0/3503520806 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f006d350 0x7f54f0151920 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 --2- 172.21.15.122:0/3503520806 >> v2:172.21.15.195:3300/0 conn(0x7f54f0141580 0x7f54f00baac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 -- 172.21.15.122:0/3503520806 >> 172.21.15.122:0/3503520806 conn(0x7f54f00bca20 msgr2=0x7f54f00bce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:34.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 -- 172.21.15.122:0/3503520806 shutdown_connections 2024-03-16T20:45:34.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 -- 172.21.15.122:0/3503520806 wait complete. 2024-03-16T20:45:34.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 Processor -- start 2024-03-16T20:45:34.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 -- start start 2024-03-16T20:45:34.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54f006d350 0x7f54f0144000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f00bb000 0x7f54f0143140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54f0141580 0x7f54f00b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f01429d0 con 0x7f54f00bb000 2024-03-16T20:45:34.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f0142210 con 0x7f54f006d350 2024-03-16T20:45:34.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f013ea90 con 0x7f54f0141580 2024-03-16T20:45:34.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54effff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f00bb000 0x7f54f0143140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54effff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f00bb000 0x7f54f0143140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34362/0 (socket says 172.21.15.122:34362) 2024-03-16T20:45:34.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54effff640 1 -- 172.21.15.122:0/837067489 learned_addr learned my addr 172.21.15.122:0/837067489 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:34.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f4b85640 1 --2- 172.21.15.122:0/837067489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54f006d350 0x7f54f0144000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f5386640 1 --2- 172.21.15.122:0/837067489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54f0141580 0x7f54f00b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54effff640 1 -- 172.21.15.122:0/837067489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54f0141580 msgr2=0x7f54f00b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54effff640 1 --2- 172.21.15.122:0/837067489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54f0141580 0x7f54f00b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54effff640 1 -- 172.21.15.122:0/837067489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54f006d350 msgr2=0x7f54f0144000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54effff640 1 --2- 172.21.15.122:0/837067489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54f006d350 0x7f54f0144000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54effff640 1 -- 172.21.15.122:0/837067489 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e8009a70 con 0x7f54f00bb000 2024-03-16T20:45:34.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54effff640 1 --2- 172.21.15.122:0/837067489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f00bb000 0x7f54f0143140 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f54e40011f0 tx=0x7f54e400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:34.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54ed7fa640 1 -- 172.21.15.122:0/837067489 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e40074e0 con 0x7f54f00bb000 2024-03-16T20:45:34.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54ed7fa640 1 -- 172.21.15.122:0/837067489 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e4007640 con 0x7f54f00bb000 2024-03-16T20:45:34.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 -- 172.21.15.122:0/837067489 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f0144830 con 0x7f54f00bb000 2024-03-16T20:45:34.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54ed7fa640 1 -- 172.21.15.122:0/837067489 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e4004a10 con 0x7f54f00bb000 2024-03-16T20:45:34.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.655+0000 7f54f6de9640 1 -- 172.21.15.122:0/837067489 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f0144540 con 0x7f54f00bb000 2024-03-16T20:45:34.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.659+0000 7f54f6de9640 1 -- 172.21.15.122:0/837067489 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f00684d0 con 0x7f54f00bb000 2024-03-16T20:45:34.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.659+0000 7f54ed7fa640 1 -- 172.21.15.122:0/837067489 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f54e4004b70 con 0x7f54f00bb000 2024-03-16T20:45:34.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.659+0000 7f54ed7fa640 1 --2- 172.21.15.122:0/837067489 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f54dc066e40 0x7f54dc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.659+0000 7f54ed7fa640 1 -- 172.21.15.122:0/837067489 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54e4083ef0 con 0x7f54f00bb000 2024-03-16T20:45:34.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.659+0000 7f54f4b85640 1 --2- 172.21.15.122:0/837067489 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f54dc066e40 0x7f54dc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.659+0000 7f54f4b85640 1 --2- 172.21.15.122:0/837067489 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f54dc066e40 0x7f54dc069300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f54f013b540 tx=0x7f54e0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:34.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.663+0000 7f54ed7fa640 1 -- 172.21.15.122:0/837067489 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f54e4050d70 con 0x7f54f00bb000 2024-03-16T20:45:34.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.847+0000 7f54f6de9640 1 -- 172.21.15.122:0/837067489 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f54f01426f0 con 0x7f54f00bb000 2024-03-16T20:45:34.852 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-03-16T20:45:34.847+0000 7f11e71dc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T20:45:34.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.851+0000 7f54ed7fa640 1 -- 172.21.15.122:0/837067489 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7f54f01426f0 con 0x7f54f00bb000 2024-03-16T20:45:34.856 INFO:teuthology.orchestra.run.smithi122.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-16T20:45:34.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.851+0000 7f54f6de9640 1 -- 172.21.15.122:0/837067489 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f54dc066e40 msgr2=0x7f54dc069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.855+0000 7f54f6de9640 1 --2- 172.21.15.122:0/837067489 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f54dc066e40 0x7f54dc069300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f54f013b540 tx=0x7f54e0008040 comp rx=0 tx=0).stop 2024-03-16T20:45:34.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.855+0000 7f54f6de9640 1 -- 172.21.15.122:0/837067489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f00bb000 msgr2=0x7f54f0143140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.855+0000 7f54f6de9640 1 --2- 172.21.15.122:0/837067489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f00bb000 0x7f54f0143140 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f54e40011f0 tx=0x7f54e400c9b0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.855+0000 7f54f6de9640 1 -- 172.21.15.122:0/837067489 shutdown_connections 2024-03-16T20:45:34.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.855+0000 7f54f6de9640 1 --2- 172.21.15.122:0/837067489 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f54dc066e40 0x7f54dc069300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.855+0000 7f54f6de9640 1 --2- 172.21.15.122:0/837067489 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f54f0141580 0x7f54f00b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.855+0000 7f54f6de9640 1 --2- 172.21.15.122:0/837067489 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54f00bb000 0x7f54f0143140 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.855+0000 7f54f6de9640 1 --2- 172.21.15.122:0/837067489 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f54f006d350 0x7f54f0144000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.855+0000 7f54f6de9640 1 -- 172.21.15.122:0/837067489 >> 172.21.15.122:0/837067489 conn(0x7f54f00bca20 msgr2=0x7f54f013fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:34.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.855+0000 7f54f6de9640 1 -- 172.21.15.122:0/837067489 shutdown_connections 2024-03-16T20:45:34.859 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.855+0000 7f54f6de9640 1 -- 172.21.15.122:0/837067489 wait complete. 2024-03-16T20:45:34.865 INFO:tasks.ceph.mds.b.smithi195.stderr: -1144> 2024-03-16T20:45:09.328+0000 7f31b2cc3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-16T20:45:34.875 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-16T20:45:34.908 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-03-16T20:45:34.984 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.979+0000 7ff5fbe52640 1 Processor -- start 2024-03-16T20:45:34.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.979+0000 7ff5fbe52640 1 -- start start 2024-03-16T20:45:34.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.979+0000 7ff5fbe52640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff5f40aef80 0x7ff5f4144390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.979+0000 7ff5fbe52640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f40b4270 con 0x7ff5f409ae30 2024-03-16T20:45:34.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.979+0000 7ff5fbe52640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f409b210 con 0x7ff5f40af410 2024-03-16T20:45:34.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.979+0000 7ff5fbe52640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f40b3110 con 0x7ff5f40aef80 2024-03-16T20:45:34.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f9bee640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7ff5f409ae30 0x7ff5f40b3b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39888/0 (socket says 172.21.15.122:39888) 2024-03-16T20:45:34.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f9bee640 1 -- 172.21.15.122:0/2167093390 learned_addr learned my addr 172.21.15.122:0/2167093390 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:34.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f93ed640 1 --2- 172.21.15.122:0/2167093390 >> v2:172.21.15.195:3300/0 conn(0x7ff5f40aef80 0x7ff5f4144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 -- 172.21.15.122:0/2167093390 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2518758680 0 0) 0x7ff5f40b4270 con 0x7ff5f409ae30 2024-03-16T20:45:34.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 -- 172.21.15.122:0/2167093390 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5e0003530 con 0x7ff5f409ae30 2024-03-16T20:45:34.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 -- 172.21.15.122:0/2167093390 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 851311847 0 0) 0x7ff5e0003530 con 0x7ff5f409ae30 2024-03-16T20:45:34.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 -- 172.21.15.122:0/2167093390 >> v2:172.21.15.195:3300/0 conn(0x7ff5f40aef80 msgr2=0x7ff5f4144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:34.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 --2- 172.21.15.122:0/2167093390 >> v2:172.21.15.195:3300/0 conn(0x7ff5f40aef80 0x7ff5f4144390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 -- 172.21.15.122:0/2167093390 >> v1:172.21.15.195:6789/0 conn(0x7ff5f40af410 legacy=0x7ff5f414e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 -- 172.21.15.122:0/2167093390 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f40b3320 con 0x7ff5f409ae30 2024-03-16T20:45:34.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 -- 172.21.15.122:0/2167093390 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7ff5f00016a0 con 0x7ff5f409ae30 2024-03-16T20:45:34.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 -- 172.21.15.122:0/2167093390 >> v1:172.21.15.122:6789/0 conn(0x7ff5f409ae30 legacy=0x7ff5f40b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 --2- 172.21.15.122:0/2167093390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5e0003980 0x7ff5e0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 --2- 172.21.15.122:0/2167093390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5e00047d0 0x7ff5e0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 --2- 172.21.15.122:0/2167093390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5e0005190 0x7ff5e000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 -- 172.21.15.122:0/2167093390 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f40b3110 con 0x7ff5e0003980 2024-03-16T20:45:34.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 -- 172.21.15.122:0/2167093390 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e000c130 con 0x7ff5e00047d0 2024-03-16T20:45:34.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 -- 172.21.15.122:0/2167093390 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e000c2f0 con 0x7ff5e0005190 2024-03-16T20:45:34.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f9bee640 1 --2- 172.21.15.122:0/2167093390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5e0003980 0x7ff5e0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5fa3ef640 1 --2- 172.21.15.122:0/2167093390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5e0005190 0x7ff5e000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f9bee640 1 -- 172.21.15.122:0/2167093390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5e0005190 msgr2=0x7ff5e000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:34.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f9bee640 1 --2- 172.21.15.122:0/2167093390 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5e0005190 0x7ff5e000b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f9bee640 1 -- 172.21.15.122:0/2167093390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5e00047d0 msgr2=0x7ff5e0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:34.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f9bee640 1 --2- 172.21.15.122:0/2167093390 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5e00047d0 0x7ff5e0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f9bee640 1 -- 172.21.15.122:0/2167093390 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e0003530 con 0x7ff5e0003980 2024-03-16T20:45:34.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5fa3ef640 1 -- 172.21.15.122:0/2167093390 reap_dead start 2024-03-16T20:45:34.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f9bee640 1 --2- 172.21.15.122:0/2167093390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5e0003980 0x7ff5e0003da0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000c190 tx=0x7ff5f001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7e9b31da0749a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:34.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 -- 172.21.15.122:0/2167093390 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0005ae0 con 0x7ff5e0003980 2024-03-16T20:45:34.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 -- 172.21.15.122:0/2167093390 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f0005c40 con 0x7ff5e0003980 2024-03-16T20:45:34.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.983+0000 7ff5f8bec640 1 -- 172.21.15.122:0/2167093390 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0007070 con 0x7ff5e0003980 2024-03-16T20:45:34.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fbe52640 1 -- 172.21.15.122:0/2167093390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5e0003980 msgr2=0x7ff5e0003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fbe52640 1 --2- 172.21.15.122:0/2167093390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5e0003980 0x7ff5e0003da0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000c190 tx=0x7ff5f001cc20 comp rx=0 tx=0).stop 2024-03-16T20:45:34.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fbe52640 1 -- 172.21.15.122:0/2167093390 shutdown_connections 2024-03-16T20:45:34.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fbe52640 1 --2- 172.21.15.122:0/2167093390 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5e0003980 0x7ff5e0003da0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fbe52640 1 -- 172.21.15.122:0/2167093390 >> 172.21.15.122:0/2167093390 conn(0x7ff5f40b8d10 msgr2=0x7ff5f409a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:34.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fbe52640 1 -- 172.21.15.122:0/2167093390 shutdown_connections 2024-03-16T20:45:34.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fbe52640 1 -- 172.21.15.122:0/2167093390 wait complete. 2024-03-16T20:45:34.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fbe52640 1 Processor -- start 2024-03-16T20:45:34.992 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fbe52640 1 -- start start 2024-03-16T20:45:34.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fbe52640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5e0003980 0x7ff5f41460a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fbe52640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5f41421a0 0x7ff5f41393e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fbe52640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f41e8010 0x7ff5f41e8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fbe52640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f4142620 con 0x7ff5f41421a0 2024-03-16T20:45:34.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fbe52640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f4145950 con 0x7ff5e0003980 2024-03-16T20:45:34.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fbe52640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f4145c10 con 0x7ff5f41e8010 2024-03-16T20:45:34.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5f93ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5f41421a0 0x7ff5f41393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5f93ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5f41421a0 0x7ff5f41393e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34378/0 (socket says 172.21.15.122:34378) 2024-03-16T20:45:34.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5f93ed640 1 -- 172.21.15.122:0/4007243691 learned_addr learned my addr 172.21.15.122:0/4007243691 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:34.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fa3ef640 1 --2- 172.21.15.122:0/4007243691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f41e8010 0x7ff5f41e8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5f93ed640 1 -- 172.21.15.122:0/4007243691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f41e8010 msgr2=0x7ff5f41e8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5f9bee640 1 --2- 172.21.15.122:0/4007243691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5e0003980 0x7ff5f41460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5f93ed640 1 --2- 172.21.15.122:0/4007243691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f41e8010 0x7ff5f41e8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5f93ed640 1 -- 172.21.15.122:0/4007243691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5e0003980 msgr2=0x7ff5f41460a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:34.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5f93ed640 1 --2- 172.21.15.122:0/4007243691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5e0003980 0x7ff5f41460a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:34.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5f93ed640 1 -- 172.21.15.122:0/4007243691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f4143970 con 0x7ff5f41421a0 2024-03-16T20:45:34.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5f93ed640 1 --2- 172.21.15.122:0/4007243691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5f41421a0 0x7ff5f41393e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400e840 tx=0x7ff5e400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:34.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5ea7fc640 1 -- 172.21.15.122:0/4007243691 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4009620 con 0x7ff5f41421a0 2024-03-16T20:45:34.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5ea7fc640 1 -- 172.21.15.122:0/4007243691 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e400c940 con 0x7ff5f41421a0 2024-03-16T20:45:34.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fbe52640 1 -- 172.21.15.122:0/4007243691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f413ce90 con 0x7ff5f41421a0 2024-03-16T20:45:34.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5ea7fc640 1 -- 172.21.15.122:0/4007243691 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e400ccf0 con 0x7ff5f41421a0 2024-03-16T20:45:34.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.987+0000 7ff5fbe52640 1 -- 172.21.15.122:0/4007243691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f4143be0 con 0x7ff5f41421a0 2024-03-16T20:45:34.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.991+0000 7ff5fbe52640 1 -- 172.21.15.122:0/4007243691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f4139f50 con 0x7ff5f41421a0 2024-03-16T20:45:34.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.991+0000 7ff5ea7fc640 1 -- 172.21.15.122:0/4007243691 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7ff5e4013470 con 0x7ff5f41421a0 2024-03-16T20:45:34.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.991+0000 7ff5ea7fc640 1 --2- 172.21.15.122:0/4007243691 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff5d8066e70 0x7ff5d8069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:34.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.991+0000 7ff5ea7fc640 1 -- 172.21.15.122:0/4007243691 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff5e4084ab0 con 0x7ff5f41421a0 2024-03-16T20:45:34.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.991+0000 7ff5f9bee640 1 --2- 172.21.15.122:0/4007243691 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff5d8066e70 0x7ff5d8069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:34.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.991+0000 7ff5f9bee640 1 --2- 172.21.15.122:0/4007243691 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff5d8066e70 0x7ff5d8069330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00ab000 tx=0x7ff5f00a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:35.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:34.995+0000 7ff5ea7fc640 1 -- 172.21.15.122:0/4007243691 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff5f4139f50 con 0x7ff5f41421a0 2024-03-16T20:45:35.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:35.191+0000 7ff5fbe52640 1 -- 172.21.15.122:0/4007243691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff5f4139f50 con 0x7ff5f41421a0 2024-03-16T20:45:35.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:35.851+0000 7ff5ea7fc640 1 -- 172.21.15.122:0/4007243691 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff5f4139f50 con 0x7ff5f41421a0 2024-03-16T20:45:35.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:35.855+0000 7ff5fbe52640 1 -- 172.21.15.122:0/4007243691 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff5d8066e70 msgr2=0x7ff5d8069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:35.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:35.855+0000 7ff5fbe52640 1 --2- 172.21.15.122:0/4007243691 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff5d8066e70 0x7ff5d8069330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00ab000 tx=0x7ff5f00a7070 comp rx=0 tx=0).stop 2024-03-16T20:45:35.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:35.855+0000 7ff5fbe52640 1 -- 172.21.15.122:0/4007243691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5f41421a0 msgr2=0x7ff5f41393e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:35.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:35.855+0000 7ff5fbe52640 1 --2- 172.21.15.122:0/4007243691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5f41421a0 0x7ff5f41393e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff5e400e840 tx=0x7ff5e400ed00 comp rx=0 tx=0).stop 2024-03-16T20:45:35.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:35.855+0000 7ff5fbe52640 1 -- 172.21.15.122:0/4007243691 shutdown_connections 2024-03-16T20:45:35.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:35.855+0000 7ff5fbe52640 1 --2- 172.21.15.122:0/4007243691 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff5d8066e70 0x7ff5d8069330 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:35.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:35.855+0000 7ff5fbe52640 1 --2- 172.21.15.122:0/4007243691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff5f41e8010 0x7ff5f41e8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:35.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:35.855+0000 7ff5fbe52640 1 --2- 172.21.15.122:0/4007243691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff5f41421a0 0x7ff5f41393e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:35.861 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:35.855+0000 7ff5fbe52640 1 --2- 172.21.15.122:0/4007243691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff5e0003980 0x7ff5f41460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:35.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:35.855+0000 7ff5fbe52640 1 -- 172.21.15.122:0/4007243691 >> 172.21.15.122:0/4007243691 conn(0x7ff5f40b8d10 msgr2=0x7ff5f40b66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:35.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:35.855+0000 7ff5fbe52640 1 -- 172.21.15.122:0/4007243691 shutdown_connections 2024-03-16T20:45:35.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:35.855+0000 7ff5fbe52640 1 -- 172.21.15.122:0/4007243691 wait complete. 2024-03-16T20:45:35.882 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-16T20:45:36.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.007+0000 7f38c052d640 1 Processor -- start 2024-03-16T20:45:36.011 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.007+0000 7f38c052d640 1 -- start start 2024-03-16T20:45:36.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.007+0000 7f38c052d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f38b809ae30 0x7f38b80b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:36.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.007+0000 7f38c052d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f38b80b7ba0 0x7f38b80b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:36.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.007+0000 7f38c052d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38b80b3fa0 con 0x7f38b81422e0 2024-03-16T20:45:36.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.007+0000 7f38c052d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b80b50a0 con 0x7f38b809ae30 2024-03-16T20:45:36.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.007+0000 7f38c052d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b80b8590 con 0x7f38b80b7ba0 2024-03-16T20:45:36.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.007+0000 7f38be2c9640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f38b809ae30 0x7f38b80b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:36.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.007+0000 7f38beaca640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f38b81422e0 0x7f38b8142770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39890/0 (socket says 172.21.15.122:39890) 2024-03-16T20:45:36.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.007+0000 7f38beaca640 1 -- 172.21.15.122:0/2643872768 learned_addr learned my addr 172.21.15.122:0/2643872768 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:36.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.007+0000 7f38bdac8640 1 --2- 172.21.15.122:0/2643872768 >> v2:172.21.15.195:3300/0 conn(0x7f38b80b7ba0 0x7f38b80b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:36.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.007+0000 7f38bd2c7640 1 -- 172.21.15.122:0/2643872768 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124307256 0 0) 0x7f38b80b3fa0 con 0x7f38b81422e0 2024-03-16T20:45:36.013 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.007+0000 7f38bd2c7640 1 -- 172.21.15.122:0/2643872768 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38a0003530 con 0x7f38b81422e0 2024-03-16T20:45:36.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.007+0000 7f38be2c9640 1 -- 172.21.15.122:0/2643872768 >> v2:172.21.15.195:3300/0 conn(0x7f38b80b7ba0 msgr2=0x7f38b80b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:36.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.007+0000 7f38be2c9640 1 --2- 172.21.15.122:0/2643872768 >> v2:172.21.15.195:3300/0 conn(0x7f38b80b7ba0 0x7f38b80b8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:36.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.007+0000 7f38be2c9640 1 -- 172.21.15.122:0/2643872768 >> v1:172.21.15.122:6789/0 conn(0x7f38b81422e0 legacy=0x7f38b8142770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:36.014 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.007+0000 7f38be2c9640 1 -- 172.21.15.122:0/2643872768 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b809b210 con 0x7f38b809ae30 2024-03-16T20:45:36.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38be2c9640 1 --2- 172.21.15.122:0/2643872768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b809ae30 0x7f38b80b4430 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f38b809e0e0 tx=0x7f38b001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6ddec0eec1e7ed server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:36.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38bd2c7640 1 -- 172.21.15.122:0/2643872768 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b00a59f0 con 0x7f38b809ae30 2024-03-16T20:45:36.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38bd2c7640 1 -- 172.21.15.122:0/2643872768 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b00a5b50 con 0x7f38b809ae30 2024-03-16T20:45:36.015 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38bd2c7640 1 -- 172.21.15.122:0/2643872768 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b00ad440 con 0x7f38b809ae30 2024-03-16T20:45:36.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38c052d640 1 -- 172.21.15.122:0/2643872768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b809ae30 msgr2=0x7f38b80b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:36.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38c052d640 1 --2- 172.21.15.122:0/2643872768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b809ae30 0x7f38b80b4430 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f38b809e0e0 tx=0x7f38b001c930 comp rx=0 tx=0).stop 2024-03-16T20:45:36.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38c052d640 1 -- 172.21.15.122:0/2643872768 shutdown_connections 2024-03-16T20:45:36.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38c052d640 1 --2- 172.21.15.122:0/2643872768 >> v2:172.21.15.195:3300/0 conn(0x7f38b80b7ba0 0x7f38b80b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:36.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38c052d640 1 --2- 172.21.15.122:0/2643872768 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b809ae30 0x7f38b80b4430 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:36.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38c052d640 1 -- 172.21.15.122:0/2643872768 >> 172.21.15.122:0/2643872768 conn(0x7f38b80b8d10 msgr2=0x7f38b809a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:36.016 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38c052d640 1 -- 172.21.15.122:0/2643872768 shutdown_connections 2024-03-16T20:45:36.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38c052d640 1 -- 172.21.15.122:0/2643872768 wait complete. 2024-03-16T20:45:36.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38c052d640 1 Processor -- start 2024-03-16T20:45:36.017 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38c052d640 1 -- start start 2024-03-16T20:45:36.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38c052d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b809ae30 0x7f38b8071900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:36.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38c052d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38b80b7ba0 0x7f38b80ae4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:36.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38c052d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38b81422e0 0x7f38b813a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:36.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38c052d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b8141a20 con 0x7f38b809ae30 2024-03-16T20:45:36.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38c052d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b813fd20 con 0x7f38b81422e0 2024-03-16T20:45:36.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38c052d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b8141c30 con 0x7f38b80b7ba0 2024-03-16T20:45:36.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38be2c9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b809ae30 0x7f38b8071900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:36.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38be2c9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b809ae30 0x7f38b8071900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34390/0 (socket says 172.21.15.122:34390) 2024-03-16T20:45:36.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.011+0000 7f38be2c9640 1 -- 172.21.15.122:0/4262372520 learned_addr learned my addr 172.21.15.122:0/4262372520 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:36.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38bdac8640 1 --2- 172.21.15.122:0/4262372520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38b80b7ba0 0x7f38b80ae4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:36.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38be2c9640 1 -- 172.21.15.122:0/4262372520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38b80b7ba0 msgr2=0x7f38b80ae4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:36.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38be2c9640 1 --2- 172.21.15.122:0/4262372520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38b80b7ba0 0x7f38b80ae4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:36.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38be2c9640 1 -- 172.21.15.122:0/4262372520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38b81422e0 msgr2=0x7f38b813a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:36.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38beaca640 1 --2- 172.21.15.122:0/4262372520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38b81422e0 0x7f38b813a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:36.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38be2c9640 1 --2- 172.21.15.122:0/4262372520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38b81422e0 0x7f38b813a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:36.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38be2c9640 1 -- 172.21.15.122:0/4262372520 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b0009a70 con 0x7f38b809ae30 2024-03-16T20:45:36.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38be2c9640 1 --2- 172.21.15.122:0/4262372520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b809ae30 0x7f38b8071900 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f38b00b0040 tx=0x7f38b00196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:36.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38b6ffd640 1 -- 172.21.15.122:0/4262372520 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b00add40 con 0x7f38b809ae30 2024-03-16T20:45:36.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38b6ffd640 1 -- 172.21.15.122:0/4262372520 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b0009450 con 0x7f38b809ae30 2024-03-16T20:45:36.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38c052d640 1 -- 172.21.15.122:0/4262372520 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b8141590 con 0x7f38b809ae30 2024-03-16T20:45:36.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38b6ffd640 1 -- 172.21.15.122:0/4262372520 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b00b7650 con 0x7f38b809ae30 2024-03-16T20:45:36.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38c052d640 1 -- 172.21.15.122:0/4262372520 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b81412b0 con 0x7f38b809ae30 2024-03-16T20:45:36.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38c052d640 1 -- 172.21.15.122:0/4262372520 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b8074b90 con 0x7f38b809ae30 2024-03-16T20:45:36.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38b6ffd640 1 -- 172.21.15.122:0/4262372520 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f38b00b4050 con 0x7f38b809ae30 2024-03-16T20:45:36.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38b6ffd640 1 --2- 172.21.15.122:0/4262372520 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f38a4066e40 0x7f38a4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:36.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38b6ffd640 1 -- 172.21.15.122:0/4262372520 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f38b011ff50 con 0x7f38b809ae30 2024-03-16T20:45:36.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38bdac8640 1 --2- 172.21.15.122:0/4262372520 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f38a4066e40 0x7f38a4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:36.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.015+0000 7f38bdac8640 1 --2- 172.21.15.122:0/4262372520 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f38a4066e40 0x7f38a4069300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f38b81e3c40 tx=0x7f38ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:36.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.019+0000 7f38b6ffd640 1 -- 172.21.15.122:0/4262372520 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f38b00ece30 con 0x7f38b809ae30 2024-03-16T20:45:36.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.167+0000 7f38c052d640 1 -- 172.21.15.122:0/4262372520 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f38b809a5d0 con 0x7f38b809ae30 2024-03-16T20:45:36.529 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T20:45:36.532 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T20:45:36.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.859+0000 7f38b6ffd640 1 -- 172.21.15.122:0/4262372520 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) v1 ==== 181+0+0 (secure 0 0 0) 0x7f38b00b20d0 con 0x7f38b809ae30 2024-03-16T20:45:36.865 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' removed 2024-03-16T20:45:36.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.863+0000 7f38c052d640 1 -- 172.21.15.122:0/4262372520 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f38a4066e40 msgr2=0x7f38a4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:36.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.863+0000 7f38c052d640 1 --2- 172.21.15.122:0/4262372520 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f38a4066e40 0x7f38a4069300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f38b81e3c40 tx=0x7f38ac008040 comp rx=0 tx=0).stop 2024-03-16T20:45:36.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.863+0000 7f38c052d640 1 -- 172.21.15.122:0/4262372520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b809ae30 msgr2=0x7f38b8071900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:36.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.863+0000 7f38c052d640 1 --2- 172.21.15.122:0/4262372520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b809ae30 0x7f38b8071900 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f38b00b0040 tx=0x7f38b00196d0 comp rx=0 tx=0).stop 2024-03-16T20:45:36.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.863+0000 7f38c052d640 1 -- 172.21.15.122:0/4262372520 shutdown_connections 2024-03-16T20:45:36.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.863+0000 7f38c052d640 1 --2- 172.21.15.122:0/4262372520 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f38a4066e40 0x7f38a4069300 secure :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f38b81e3c40 tx=0x7f38ac008040 comp rx=0 tx=0).stop 2024-03-16T20:45:36.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.863+0000 7f38c052d640 1 --2- 172.21.15.122:0/4262372520 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f38b81422e0 0x7f38b813a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:36.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.863+0000 7f38c052d640 1 --2- 172.21.15.122:0/4262372520 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f38b80b7ba0 0x7f38b80ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:36.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.863+0000 7f38c052d640 1 --2- 172.21.15.122:0/4262372520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f38b809ae30 0x7f38b8071900 secure :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f38b00b0040 tx=0x7f38b00196d0 comp rx=0 tx=0).stop 2024-03-16T20:45:36.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.863+0000 7f38c052d640 1 -- 172.21.15.122:0/4262372520 >> 172.21.15.122:0/4262372520 conn(0x7f38b80b8d10 msgr2=0x7f38b806d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:36.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.867+0000 7f38c052d640 1 -- 172.21.15.122:0/4262372520 shutdown_connections 2024-03-16T20:45:36.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:36.867+0000 7f38c052d640 1 -- 172.21.15.122:0/4262372520 wait complete. 2024-03-16T20:45:36.895 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-16T20:45:37.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.015+0000 7f9456187640 1 Processor -- start 2024-03-16T20:45:37.019 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.015+0000 7f9456187640 1 -- start start 2024-03-16T20:45:37.020 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.015+0000 7f9456187640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f945009ae30 0x7f94500b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:37.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.015+0000 7f9456187640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94500b7ba0 0x7f94500b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:37.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.015+0000 7f9456187640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94500b3fa0 con 0x7f94501422e0 2024-03-16T20:45:37.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.015+0000 7f9456187640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94500b50a0 con 0x7f945009ae30 2024-03-16T20:45:37.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.015+0000 7f9456187640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94500b8560 con 0x7f94500b7ba0 2024-03-16T20:45:37.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.015+0000 7f944f7fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f945009ae30 0x7f94500b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:37.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.015+0000 7f944ffff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f94501422e0 0x7f9450142730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39896/0 (socket says 172.21.15.122:39896) 2024-03-16T20:45:37.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.015+0000 7f944ffff640 1 -- 172.21.15.122:0/1353052030 learned_addr learned my addr 172.21.15.122:0/1353052030 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:37.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.015+0000 7f944effd640 1 --2- 172.21.15.122:0/1353052030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94500b7ba0 0x7f94500b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:37.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.015+0000 7f944f7fe640 1 -- 172.21.15.122:0/1353052030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94500b7ba0 msgr2=0x7f94500b8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:37.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.015+0000 7f944f7fe640 1 --2- 172.21.15.122:0/1353052030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94500b7ba0 0x7f94500b8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:37.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.015+0000 7f944f7fe640 1 -- 172.21.15.122:0/1353052030 >> v1:172.21.15.122:6789/0 conn(0x7f94501422e0 legacy=0x7f9450142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:37.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.015+0000 7f944e7fc640 1 -- 172.21.15.122:0/1353052030 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331629350 0 0) 0x7f94500b3fa0 con 0x7f94501422e0 2024-03-16T20:45:37.021 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.015+0000 7f944f7fe640 1 -- 172.21.15.122:0/1353052030 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945009b210 con 0x7f945009ae30 2024-03-16T20:45:37.022 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.015+0000 7f944effd640 1 --2- 172.21.15.122:0/1353052030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94500b7ba0 0x7f94500b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:37.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f944f7fe640 1 --2- 172.21.15.122:0/1353052030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f945009ae30 0x7f94500b4430 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9448009d90 tx=0x7f944801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21aa19b54653b326 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:37.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f944e7fc640 1 -- 172.21.15.122:0/1353052030 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94480a5a30 con 0x7f945009ae30 2024-03-16T20:45:37.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f944e7fc640 1 -- 172.21.15.122:0/1353052030 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94480a5b90 con 0x7f945009ae30 2024-03-16T20:45:37.023 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f944e7fc640 1 -- 172.21.15.122:0/1353052030 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94480ad440 con 0x7f945009ae30 2024-03-16T20:45:37.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f9456187640 1 -- 172.21.15.122:0/1353052030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f945009ae30 msgr2=0x7f94500b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:37.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f9456187640 1 --2- 172.21.15.122:0/1353052030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f945009ae30 0x7f94500b4430 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9448009d90 tx=0x7f944801c970 comp rx=0 tx=0).stop 2024-03-16T20:45:37.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f9456187640 1 -- 172.21.15.122:0/1353052030 shutdown_connections 2024-03-16T20:45:37.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f9456187640 1 --2- 172.21.15.122:0/1353052030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f94500b7ba0 0x7f94500b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:37.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f9456187640 1 --2- 172.21.15.122:0/1353052030 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f945009ae30 0x7f94500b4430 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:37.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f9456187640 1 -- 172.21.15.122:0/1353052030 >> 172.21.15.122:0/1353052030 conn(0x7f94500b8d10 msgr2=0x7f945009a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:37.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f9456187640 1 -- 172.21.15.122:0/1353052030 shutdown_connections 2024-03-16T20:45:37.024 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f9456187640 1 -- 172.21.15.122:0/1353052030 wait complete. 2024-03-16T20:45:37.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f9456187640 1 Processor -- start 2024-03-16T20:45:37.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f9456187640 1 -- start start 2024-03-16T20:45:37.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f9456187640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f945009ae30 0x7f9450140e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:37.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f9456187640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94500b7ba0 0x7f9450142ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:37.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f9456187640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94501422e0 0x7f94500b0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:37.027 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f9456187640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94501440c0 con 0x7f94500b7ba0 2024-03-16T20:45:37.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f9456187640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9450144c60 con 0x7f94501422e0 2024-03-16T20:45:37.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f9456187640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94500b11f0 con 0x7f945009ae30 2024-03-16T20:45:37.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f944effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94500b7ba0 0x7f9450142ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:37.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f944effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94500b7ba0 0x7f9450142ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34412/0 (socket says 172.21.15.122:34412) 2024-03-16T20:45:37.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f944effd640 1 -- 172.21.15.122:0/1210285547 learned_addr learned my addr 172.21.15.122:0/1210285547 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:37.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f944ffff640 1 --2- 172.21.15.122:0/1210285547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94501422e0 0x7f94500b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:37.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f944f7fe640 1 --2- 172.21.15.122:0/1210285547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f945009ae30 0x7f9450140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:37.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f944effd640 1 -- 172.21.15.122:0/1210285547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f945009ae30 msgr2=0x7f9450140e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:37.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f944effd640 1 --2- 172.21.15.122:0/1210285547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f945009ae30 0x7f9450140e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:37.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f944effd640 1 -- 172.21.15.122:0/1210285547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94501422e0 msgr2=0x7f94500b0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:37.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f944effd640 1 --2- 172.21.15.122:0/1210285547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94501422e0 0x7f94500b0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:37.028 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f944effd640 1 -- 172.21.15.122:0/1210285547 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9448009a70 con 0x7f94500b7ba0 2024-03-16T20:45:37.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.019+0000 7f944ffff640 1 --2- 172.21.15.122:0/1210285547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94501422e0 0x7f94500b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:37.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.023+0000 7f944effd640 1 --2- 172.21.15.122:0/1210285547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94500b7ba0 0x7f9450142ea0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f944400c7c0 tx=0x7f944400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:37.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.023+0000 7f9437fff640 1 -- 172.21.15.122:0/1210285547 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444007a30 con 0x7f94500b7ba0 2024-03-16T20:45:37.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.023+0000 7f9437fff640 1 -- 172.21.15.122:0/1210285547 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9444007b90 con 0x7f94500b7ba0 2024-03-16T20:45:37.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.023+0000 7f9456187640 1 -- 172.21.15.122:0/1210285547 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94500b0190 con 0x7f94500b7ba0 2024-03-16T20:45:37.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.023+0000 7f9437fff640 1 -- 172.21.15.122:0/1210285547 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944400ce70 con 0x7f94500b7ba0 2024-03-16T20:45:37.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.023+0000 7f9456187640 1 -- 172.21.15.122:0/1210285547 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94500afe20 con 0x7f94500b7ba0 2024-03-16T20:45:37.029 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.023+0000 7f9456187640 1 -- 172.21.15.122:0/1210285547 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9450144df0 con 0x7f94500b7ba0 2024-03-16T20:45:37.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.023+0000 7f9437fff640 1 -- 172.21.15.122:0/1210285547 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f94440113f0 con 0x7f94500b7ba0 2024-03-16T20:45:37.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.023+0000 7f9437fff640 1 --2- 172.21.15.122:0/1210285547 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f943c066e40 0x7f943c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:37.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.023+0000 7f9437fff640 1 -- 172.21.15.122:0/1210285547 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f9444084e10 con 0x7f94500b7ba0 2024-03-16T20:45:37.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.023+0000 7f944f7fe640 1 --2- 172.21.15.122:0/1210285547 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f943c066e40 0x7f943c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:37.030 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.023+0000 7f944f7fe640 1 --2- 172.21.15.122:0/1210285547 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f943c066e40 0x7f943c069300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f94480af000 tx=0x7f94480a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:37.032 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.027+0000 7f9437fff640 1 -- 172.21.15.122:0/1210285547 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9444051ed0 con 0x7f94500b7ba0 2024-03-16T20:45:37.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.187+0000 7f9456187640 1 -- 172.21.15.122:0/1210285547 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9450074bb0 con 0x7f94500b7ba0 2024-03-16T20:45:37.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.867+0000 7f9437fff640 1 -- 172.21.15.122:0/1210285547 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9444055590 con 0x7f94500b7ba0 2024-03-16T20:45:37.874 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' removed 2024-03-16T20:45:37.877 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.871+0000 7f9456187640 1 -- 172.21.15.122:0/1210285547 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f943c066e40 msgr2=0x7f943c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:37.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.871+0000 7f9456187640 1 --2- 172.21.15.122:0/1210285547 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f943c066e40 0x7f943c069300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f94480af000 tx=0x7f94480a7070 comp rx=0 tx=0).stop 2024-03-16T20:45:37.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.871+0000 7f9456187640 1 -- 172.21.15.122:0/1210285547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94500b7ba0 msgr2=0x7f9450142ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:37.878 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.871+0000 7f9456187640 1 --2- 172.21.15.122:0/1210285547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94500b7ba0 0x7f9450142ea0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f944400c7c0 tx=0x7f944400cc80 comp rx=0 tx=0).stop 2024-03-16T20:45:37.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.875+0000 7f9456187640 1 -- 172.21.15.122:0/1210285547 shutdown_connections 2024-03-16T20:45:37.880 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.875+0000 7f9456187640 1 --2- 172.21.15.122:0/1210285547 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f943c066e40 0x7f943c069300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:37.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.875+0000 7f9456187640 1 --2- 172.21.15.122:0/1210285547 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f94501422e0 0x7f94500b0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:37.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.875+0000 7f9456187640 1 --2- 172.21.15.122:0/1210285547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f94500b7ba0 0x7f9450142ea0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:37.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.875+0000 7f9456187640 1 --2- 172.21.15.122:0/1210285547 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f945009ae30 0x7f9450140e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:37.881 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.875+0000 7f9456187640 1 -- 172.21.15.122:0/1210285547 >> 172.21.15.122:0/1210285547 conn(0x7f94500b8d10 msgr2=0x7f9450099990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:37.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.875+0000 7f9456187640 1 -- 172.21.15.122:0/1210285547 shutdown_connections 2024-03-16T20:45:37.882 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:37.879+0000 7f9456187640 1 -- 172.21.15.122:0/1210285547 wait complete. 2024-03-16T20:45:37.905 INFO:tasks.ceph.mds.a:Restarting daemon 2024-03-16T20:45:37.906 INFO:tasks.ceph.mds.a:Stopping old one... 2024-03-16T20:45:37.906 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-16T20:45:37.906 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:45:37.906 INFO:tasks.ceph.mds.b:Restarting daemon 2024-03-16T20:45:37.906 INFO:tasks.ceph.mds.b:Stopping old one... 2024-03-16T20:45:37.907 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-16T20:45:37.907 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:45:37.915 INFO:tasks.ceph.mds.a:Stopped 2024-03-16T20:45:37.916 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-03-16T20:45:37.924 INFO:tasks.ceph.mds.b:Stopped 2024-03-16T20:45:37.924 DEBUG:teuthology.orchestra.run.smithi195:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-03-16T20:45:37.963 INFO:tasks.ceph.mds.a:Started 2024-03-16T20:45:37.964 DEBUG:teuthology.parallel:result is None 2024-03-16T20:45:37.974 INFO:tasks.ceph.mds.b:Started 2024-03-16T20:45:37.974 DEBUG:teuthology.parallel:result is None 2024-03-16T20:45:37.974 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-03-16T20:45:38.031 INFO:tasks.ceph.mds.a.smithi122.stdout:starting mds.a at 2024-03-16T20:45:38.044 INFO:tasks.ceph.mds.b.smithi195.stdout:starting mds.b at 2024-03-16T20:45:38.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.087+0000 7f209d8f6640 1 Processor -- start 2024-03-16T20:45:38.094 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.087+0000 7f209d8f6640 1 -- start start 2024-03-16T20:45:38.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f209d8f6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20980b82b0 0x7f20980b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:38.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f209d8f6640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f20981422e0 0x7f2098142740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:38.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f209d8f6640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20980b3fa0 con 0x7f209809ae30 2024-03-16T20:45:38.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f209d8f6640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20980b50a0 con 0x7f20981422e0 2024-03-16T20:45:38.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f209d8f6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209809b210 con 0x7f20980b82b0 2024-03-16T20:45:38.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f2096ffd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f209809ae30 0x7f20980b7ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39912/0 (socket says 172.21.15.122:39912) 2024-03-16T20:45:38.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f2096ffd640 1 -- 172.21.15.122:0/4244033671 learned_addr learned my addr 172.21.15.122:0/4244033671 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:38.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f20977fe640 1 --2- 172.21.15.122:0/4244033671 >> v2:172.21.15.122:3300/0 conn(0x7f20981422e0 0x7f2098142740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:38.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f20967fc640 1 --2- 172.21.15.122:0/4244033671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20980b82b0 0x7f20980b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:38.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f20977fe640 1 -- 172.21.15.122:0/4244033671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20980b82b0 msgr2=0x7f20980b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:38.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f20977fe640 1 --2- 172.21.15.122:0/4244033671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20980b82b0 0x7f20980b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:38.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f20977fe640 1 -- 172.21.15.122:0/4244033671 >> v1:172.21.15.122:6789/0 conn(0x7f209809ae30 legacy=0x7f20980b7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:38.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f20977fe640 1 -- 172.21.15.122:0/4244033671 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2098142c80 con 0x7f20981422e0 2024-03-16T20:45:38.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f20977fe640 1 --2- 172.21.15.122:0/4244033671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20981422e0 0x7f2098142740 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2090009d90 tx=0x7f209001c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93589e7bf0b9a687 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:38.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f2095ffb640 1 -- 172.21.15.122:0/4244033671 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900a5800 con 0x7f20981422e0 2024-03-16T20:45:38.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f2095ffb640 1 -- 172.21.15.122:0/4244033671 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20900a5960 con 0x7f20981422e0 2024-03-16T20:45:38.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f2095ffb640 1 -- 172.21.15.122:0/4244033671 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900a5cf0 con 0x7f20981422e0 2024-03-16T20:45:38.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f209d8f6640 1 -- 172.21.15.122:0/4244033671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20981422e0 msgr2=0x7f2098142740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:38.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f209d8f6640 1 --2- 172.21.15.122:0/4244033671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20981422e0 0x7f2098142740 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2090009d90 tx=0x7f209001c800 comp rx=0 tx=0).stop 2024-03-16T20:45:38.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f209d8f6640 1 -- 172.21.15.122:0/4244033671 shutdown_connections 2024-03-16T20:45:38.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f209d8f6640 1 --2- 172.21.15.122:0/4244033671 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20981422e0 0x7f2098142740 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:38.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f209d8f6640 1 --2- 172.21.15.122:0/4244033671 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f20980b82b0 0x7f20980b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:38.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f209d8f6640 1 -- 172.21.15.122:0/4244033671 >> 172.21.15.122:0/4244033671 conn(0x7f20980b8d10 msgr2=0x7f209809a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:38.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.091+0000 7f209d8f6640 1 -- 172.21.15.122:0/4244033671 shutdown_connections 2024-03-16T20:45:38.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f209d8f6640 1 -- 172.21.15.122:0/4244033671 wait complete. 2024-03-16T20:45:38.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f209d8f6640 1 Processor -- start 2024-03-16T20:45:38.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f209d8f6640 1 -- start start 2024-03-16T20:45:38.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f209d8f6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f209809ae30 0x7f2098141410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:38.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f209d8f6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20980b82b0 0x7f20980977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:38.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f209d8f6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20981422e0 0x7f20981dd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:38.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f209d8f6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209814f540 con 0x7f20980b82b0 2024-03-16T20:45:38.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f209d8f6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209813e900 con 0x7f20981422e0 2024-03-16T20:45:38.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f209d8f6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2098140ca0 con 0x7f209809ae30 2024-03-16T20:45:38.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f20967fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20980b82b0 0x7f20980977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:38.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f20967fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20980b82b0 0x7f20980977a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:34450/0 (socket says 172.21.15.122:34450) 2024-03-16T20:45:38.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f20967fc640 1 -- 172.21.15.122:0/773316071 learned_addr learned my addr 172.21.15.122:0/773316071 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:38.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f20977fe640 1 --2- 172.21.15.122:0/773316071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20981422e0 0x7f20981dd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:38.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f2096ffd640 1 --2- 172.21.15.122:0/773316071 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f209809ae30 0x7f2098141410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:38.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f20967fc640 1 -- 172.21.15.122:0/773316071 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f209809ae30 msgr2=0x7f2098141410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:38.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f20967fc640 1 --2- 172.21.15.122:0/773316071 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f209809ae30 0x7f2098141410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:38.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f20967fc640 1 -- 172.21.15.122:0/773316071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20981422e0 msgr2=0x7f20981dd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:38.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f20967fc640 1 --2- 172.21.15.122:0/773316071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20981422e0 0x7f20981dd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:38.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f20967fc640 1 -- 172.21.15.122:0/773316071 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2090009a70 con 0x7f20980b82b0 2024-03-16T20:45:38.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f20967fc640 1 --2- 172.21.15.122:0/773316071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20980b82b0 0x7f20980977a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f208c00e980 tx=0x7f208c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:38.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f207ffff640 1 -- 172.21.15.122:0/773316071 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c009670 con 0x7f20980b82b0 2024-03-16T20:45:38.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f207ffff640 1 -- 172.21.15.122:0/773316071 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208c00ca00 con 0x7f20980b82b0 2024-03-16T20:45:38.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f209d8f6640 1 -- 172.21.15.122:0/773316071 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2098140fe0 con 0x7f20980b82b0 2024-03-16T20:45:38.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f207ffff640 1 -- 172.21.15.122:0/773316071 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208c00cd90 con 0x7f20980b82b0 2024-03-16T20:45:38.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f209d8f6640 1 -- 172.21.15.122:0/773316071 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2098140800 con 0x7f20980b82b0 2024-03-16T20:45:38.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f209d8f6640 1 -- 172.21.15.122:0/773316071 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2098140a50 con 0x7f20980b82b0 2024-03-16T20:45:38.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.095+0000 7f207ffff640 1 -- 172.21.15.122:0/773316071 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f208c013470 con 0x7f20980b82b0 2024-03-16T20:45:38.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.099+0000 7f207ffff640 1 --2- 172.21.15.122:0/773316071 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2084066e40 0x7f2084069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:38.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.099+0000 7f207ffff640 1 -- 172.21.15.122:0/773316071 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f208c0841e0 con 0x7f20980b82b0 2024-03-16T20:45:38.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.099+0000 7f2096ffd640 1 --2- 172.21.15.122:0/773316071 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2084066e40 0x7f2084069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:38.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.099+0000 7f2096ffd640 1 --2- 172.21.15.122:0/773316071 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2084066e40 0x7f2084069300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2088007fd0 tx=0x7f2088009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:38.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.103+0000 7f207ffff640 1 -- 172.21.15.122:0/773316071 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f208c051410 con 0x7f20980b82b0 2024-03-16T20:45:38.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.271+0000 7f209d8f6640 1 -- 172.21.15.122:0/773316071 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2098074bb0 con 0x7f20980b82b0 2024-03-16T20:45:38.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.891+0000 7f207ffff640 1 -- 172.21.15.122:0/773316071 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) v1 ==== 97+0+0 (secure 0 0 0) 0x7f208c054ad0 con 0x7f20980b82b0 2024-03-16T20:45:38.897 INFO:teuthology.orchestra.run.smithi122.stderr: removed all blocklist entries 2024-03-16T20:45:38.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.895+0000 7f209d8f6640 1 -- 172.21.15.122:0/773316071 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2084066e40 msgr2=0x7f2084069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:38.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.895+0000 7f209d8f6640 1 --2- 172.21.15.122:0/773316071 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2084066e40 0x7f2084069300 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2088007fd0 tx=0x7f2088009040 comp rx=0 tx=0).stop 2024-03-16T20:45:38.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.895+0000 7f209d8f6640 1 -- 172.21.15.122:0/773316071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20980b82b0 msgr2=0x7f20980977a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:38.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.895+0000 7f209d8f6640 1 --2- 172.21.15.122:0/773316071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20980b82b0 0x7f20980977a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f208c00e980 tx=0x7f208c00ee40 comp rx=0 tx=0).stop 2024-03-16T20:45:38.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.895+0000 7f209d8f6640 1 -- 172.21.15.122:0/773316071 shutdown_connections 2024-03-16T20:45:38.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.895+0000 7f209d8f6640 1 --2- 172.21.15.122:0/773316071 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2084066e40 0x7f2084069300 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:38.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.895+0000 7f209d8f6640 1 --2- 172.21.15.122:0/773316071 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f20981422e0 0x7f20981dd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:38.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.895+0000 7f209d8f6640 1 --2- 172.21.15.122:0/773316071 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20980b82b0 0x7f20980977a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:38.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.895+0000 7f209d8f6640 1 --2- 172.21.15.122:0/773316071 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f209809ae30 0x7f2098141410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:38.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.895+0000 7f209d8f6640 1 -- 172.21.15.122:0/773316071 >> 172.21.15.122:0/773316071 conn(0x7f20980b8d10 msgr2=0x7f209806d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:38.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.895+0000 7f209d8f6640 1 -- 172.21.15.122:0/773316071 shutdown_connections 2024-03-16T20:45:38.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:38.895+0000 7f209d8f6640 1 -- 172.21.15.122:0/773316071 wait complete. 2024-03-16T20:45:38.919 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-03-16T20:45:39.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.031+0000 7f356f1fd640 1 Processor -- start 2024-03-16T20:45:39.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.031+0000 7f356f1fd640 1 -- start start 2024-03-16T20:45:39.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f356f1fd640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f356809ba00 0x7f35681471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f356f1fd640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356809aee0 con 0x7f3568147e90 2024-03-16T20:45:39.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f356f1fd640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f356809ab70 con 0x7f35680ba850 2024-03-16T20:45:39.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f356f1fd640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35680bb1d0 con 0x7f356809ba00 2024-03-16T20:45:39.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f356cf99640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f3568147e90 0x7f35680b8b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39444/0 (socket says 172.21.15.122:39444) 2024-03-16T20:45:39.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f356cf99640 1 -- 172.21.15.122:0/1010929223 learned_addr learned my addr 172.21.15.122:0/1010929223 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:39.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f356d79a640 1 --2- 172.21.15.122:0/1010929223 >> v2:172.21.15.195:3300/0 conn(0x7f356809ba00 0x7f35681471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.040 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f35677fe640 1 -- 172.21.15.122:0/1010929223 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51636046 0 0) 0x7f356809aee0 con 0x7f3568147e90 2024-03-16T20:45:39.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f35677fe640 1 -- 172.21.15.122:0/1010929223 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3550003530 con 0x7f3568147e90 2024-03-16T20:45:39.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f35677fe640 1 -- 172.21.15.122:0/1010929223 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2003689774 0 0) 0x7f3550003530 con 0x7f3568147e90 2024-03-16T20:45:39.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f35677fe640 1 -- 172.21.15.122:0/1010929223 >> v2:172.21.15.195:3300/0 conn(0x7f356809ba00 msgr2=0x7f35681471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:39.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f35677fe640 1 --2- 172.21.15.122:0/1010929223 >> v2:172.21.15.195:3300/0 conn(0x7f356809ba00 0x7f35681471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f35677fe640 1 -- 172.21.15.122:0/1010929223 >> v1:172.21.15.195:6789/0 conn(0x7f35680ba850 legacy=0x7f356814fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:39.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f35677fe640 1 -- 172.21.15.122:0/1010929223 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356809c150 con 0x7f3568147e90 2024-03-16T20:45:39.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f35677fe640 1 -- 172.21.15.122:0/1010929223 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f355c001680 con 0x7f3568147e90 2024-03-16T20:45:39.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f35677fe640 1 -- 172.21.15.122:0/1010929223 >> v1:172.21.15.122:6789/0 conn(0x7f3568147e90 legacy=0x7f35680b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:39.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f35677fe640 1 --2- 172.21.15.122:0/1010929223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3550003980 0x7f3550003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f35677fe640 1 --2- 172.21.15.122:0/1010929223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35500047d0 0x7f3550004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f35677fe640 1 --2- 172.21.15.122:0/1010929223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35500052b0 0x7f355000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f35677fe640 1 -- 172.21.15.122:0/1010929223 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35680bb1d0 con 0x7f3550003980 2024-03-16T20:45:39.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f35677fe640 1 -- 172.21.15.122:0/1010929223 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355000c180 con 0x7f35500047d0 2024-03-16T20:45:39.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f35677fe640 1 -- 172.21.15.122:0/1010929223 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f355000c340 con 0x7f35500052b0 2024-03-16T20:45:39.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f356cf99640 1 --2- 172.21.15.122:0/1010929223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3550003980 0x7f3550003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f3567fff640 1 --2- 172.21.15.122:0/1010929223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35500047d0 0x7f3550004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f356d79a640 1 --2- 172.21.15.122:0/1010929223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35500052b0 0x7f355000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f356cf99640 1 -- 172.21.15.122:0/1010929223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35500052b0 msgr2=0x7f355000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:39.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f356cf99640 1 --2- 172.21.15.122:0/1010929223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35500052b0 0x7f355000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f356cf99640 1 -- 172.21.15.122:0/1010929223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35500047d0 msgr2=0x7f3550004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:39.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f356cf99640 1 --2- 172.21.15.122:0/1010929223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35500047d0 0x7f3550004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f356cf99640 1 -- 172.21.15.122:0/1010929223 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3550003530 con 0x7f3550003980 2024-03-16T20:45:39.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f3567fff640 1 --2- 172.21.15.122:0/1010929223 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f35500047d0 0x7f3550004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:39.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f356d79a640 1 --2- 172.21.15.122:0/1010929223 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35500052b0 0x7f355000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:39.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.035+0000 7f356d79a640 1 -- 172.21.15.122:0/1010929223 reap_dead start 2024-03-16T20:45:39.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356cf99640 1 --2- 172.21.15.122:0/1010929223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3550003980 0x7f3550003da0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f355c00c210 tx=0x7f355c01cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2d18706c8d61df server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:39.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f35677fe640 1 -- 172.21.15.122:0/1010929223 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c005ae0 con 0x7f3550003980 2024-03-16T20:45:39.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f35677fe640 1 -- 172.21.15.122:0/1010929223 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355c005c40 con 0x7f3550003980 2024-03-16T20:45:39.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f35677fe640 1 -- 172.21.15.122:0/1010929223 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c0072b0 con 0x7f3550003980 2024-03-16T20:45:39.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356f1fd640 1 -- 172.21.15.122:0/1010929223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3550003980 msgr2=0x7f3550003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:39.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356f1fd640 1 --2- 172.21.15.122:0/1010929223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3550003980 0x7f3550003da0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f355c00c210 tx=0x7f355c01cb60 comp rx=0 tx=0).stop 2024-03-16T20:45:39.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356f1fd640 1 -- 172.21.15.122:0/1010929223 shutdown_connections 2024-03-16T20:45:39.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356f1fd640 1 --2- 172.21.15.122:0/1010929223 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3550003980 0x7f3550003da0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356f1fd640 1 -- 172.21.15.122:0/1010929223 >> 172.21.15.122:0/1010929223 conn(0x7f35680bca50 msgr2=0x7f35680bce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:39.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356f1fd640 1 -- 172.21.15.122:0/1010929223 shutdown_connections 2024-03-16T20:45:39.044 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356f1fd640 1 -- 172.21.15.122:0/1010929223 wait complete. 2024-03-16T20:45:39.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356f1fd640 1 Processor -- start 2024-03-16T20:45:39.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356f1fd640 1 -- start start 2024-03-16T20:45:39.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356f1fd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3550003980 0x7f35681431a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356f1fd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3568142660 0x7f35680a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356f1fd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35680a0c70 0x7f3568140c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356f1fd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35680b3fb0 con 0x7f3550003980 2024-03-16T20:45:39.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356f1fd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35680b6e80 con 0x7f3568142660 2024-03-16T20:45:39.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356f1fd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3568143a60 con 0x7f35680a0c70 2024-03-16T20:45:39.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356cf99640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3550003980 0x7f35681431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356cf99640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3550003980 0x7f35681431a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42106/0 (socket says 172.21.15.122:42106) 2024-03-16T20:45:39.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356cf99640 1 -- 172.21.15.122:0/1629164966 learned_addr learned my addr 172.21.15.122:0/1629164966 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:39.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f3567fff640 1 --2- 172.21.15.122:0/1629164966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3568142660 0x7f35680a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356d79a640 1 --2- 172.21.15.122:0/1629164966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35680a0c70 0x7f3568140c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356cf99640 1 -- 172.21.15.122:0/1629164966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35680a0c70 msgr2=0x7f3568140c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:39.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356cf99640 1 --2- 172.21.15.122:0/1629164966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35680a0c70 0x7f3568140c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356cf99640 1 -- 172.21.15.122:0/1629164966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3568142660 msgr2=0x7f35680a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:39.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356cf99640 1 --2- 172.21.15.122:0/1629164966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3568142660 0x7f35680a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356cf99640 1 -- 172.21.15.122:0/1629164966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3568143f10 con 0x7f3550003980 2024-03-16T20:45:39.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f356d79a640 1 --2- 172.21.15.122:0/1629164966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35680a0c70 0x7f3568140c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:39.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.039+0000 7f3567fff640 1 --2- 172.21.15.122:0/1629164966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3568142660 0x7f35680a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:39.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.043+0000 7f356cf99640 1 --2- 172.21.15.122:0/1629164966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3550003980 0x7f35681431a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f355c01c5d0 tx=0x7f3550005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:39.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.043+0000 7f35657fa640 1 -- 172.21.15.122:0/1629164966 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c0b4070 con 0x7f3550003980 2024-03-16T20:45:39.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.043+0000 7f35657fa640 1 -- 172.21.15.122:0/1629164966 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355c007070 con 0x7f3550003980 2024-03-16T20:45:39.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.043+0000 7f35657fa640 1 -- 172.21.15.122:0/1629164966 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355c007400 con 0x7f3550003980 2024-03-16T20:45:39.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.043+0000 7f356f1fd640 1 -- 172.21.15.122:0/1629164966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35680a21e0 con 0x7f3550003980 2024-03-16T20:45:39.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.043+0000 7f356f1fd640 1 -- 172.21.15.122:0/1629164966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35680a2460 con 0x7f3550003980 2024-03-16T20:45:39.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.043+0000 7f356f1fd640 1 -- 172.21.15.122:0/1629164966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3568075280 con 0x7f3550003980 2024-03-16T20:45:39.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.043+0000 7f35657fa640 1 -- 172.21.15.122:0/1629164966 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f355c007560 con 0x7f3550003980 2024-03-16T20:45:39.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.043+0000 7f35657fa640 1 --2- 172.21.15.122:0/1629164966 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3554066e40 0x7f3554069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.043+0000 7f35657fa640 1 -- 172.21.15.122:0/1629164966 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f355c11dba0 con 0x7f3550003980 2024-03-16T20:45:39.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.043+0000 7f3567fff640 1 --2- 172.21.15.122:0/1629164966 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3554066e40 0x7f3554069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.043+0000 7f3567fff640 1 --2- 172.21.15.122:0/1629164966 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3554066e40 0x7f3554069300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3558000c10 tx=0x7f3558009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:39.052 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.047+0000 7f35657fa640 1 -- 172.21.15.122:0/1629164966 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f355c0eac40 con 0x7f3550003980 2024-03-16T20:45:39.279 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.275+0000 7f356f1fd640 1 -- 172.21.15.122:0/1629164966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3568144bf0 con 0x7f3550003980 2024-03-16T20:45:39.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.275+0000 7f35657fa640 1 -- 172.21.15.122:0/1629164966 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+5207 (secure 0 0 0) 0x7f355c0ee300 con 0x7f3550003980 2024-03-16T20:45:39.281 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:39.281 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-16T20:45:39.281 INFO:teuthology.orchestra.run.smithi122.stdout: "auth_dump": [ 2024-03-16T20:45:39.281 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.281 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.a", 2024-03-16T20:45:39.281 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCYBPZldiQwJxAAqp05o6yqMhy6pZoNHnDHKA==", 2024-03-16T20:45:39.281 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.281 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-03-16T20:45:39.281 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-03-16T20:45:39.282 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-03-16T20:45:39.282 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-03-16T20:45:39.282 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.282 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.282 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.282 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mds.b", 2024-03-16T20:45:39.282 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCYBPZlxwYaLRAAY7PpU3RdRuEvXt9RpPEWiQ==", 2024-03-16T20:45:39.282 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.282 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-03-16T20:45:39.282 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-03-16T20:45:39.282 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-03-16T20:45:39.283 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-03-16T20:45:39.283 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.283 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.0", 2024-03-16T20:45:39.283 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCbBPZlS35pLxAAh2wHT8Aeinet1hO5A1ioyg==", 2024-03-16T20:45:39.283 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.283 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-03-16T20:45:39.283 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-03-16T20:45:39.283 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-03-16T20:45:39.283 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.283 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.283 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.283 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.1", 2024-03-16T20:45:39.284 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCeBPZl+cEDNRAAPCKGGs2VdPWHsPvgdOnxIQ==", 2024-03-16T20:45:39.284 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.284 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-03-16T20:45:39.284 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-03-16T20:45:39.284 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-03-16T20:45:39.284 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.284 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.284 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.284 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.2", 2024-03-16T20:45:39.284 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQChBPZlYcdGOxAAW+DGd7sDp76oQ7YgETtrBg==", 2024-03-16T20:45:39.284 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.284 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-03-16T20:45:39.284 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-03-16T20:45:39.284 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-03-16T20:45:39.285 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.285 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.3", 2024-03-16T20:45:39.285 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQClBPZleeJnCRAAueEygVrApZN0Cc6L2q2kng==", 2024-03-16T20:45:39.285 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.285 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-03-16T20:45:39.285 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-03-16T20:45:39.285 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-03-16T20:45:39.285 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.285 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.285 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.285 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.4", 2024-03-16T20:45:39.286 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCpBPZlvYwKDhAAZ5t2Tj/h1Fwnqsto01FtSg==", 2024-03-16T20:45:39.286 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.286 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-03-16T20:45:39.286 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-03-16T20:45:39.286 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-03-16T20:45:39.286 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.286 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.286 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.286 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.5", 2024-03-16T20:45:39.286 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCsBPZlr1gqEhAALvShoDQpoXGkzrJCAknq+A==", 2024-03-16T20:45:39.286 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.286 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-03-16T20:45:39.286 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-03-16T20:45:39.287 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-03-16T20:45:39.287 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.287 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.6", 2024-03-16T20:45:39.287 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCvBPZlzlQuGBAAgqfkx5fzOYjnokkR+UPOpQ==", 2024-03-16T20:45:39.287 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.287 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-03-16T20:45:39.287 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-03-16T20:45:39.287 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-03-16T20:45:39.287 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.287 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.287 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.287 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "osd.7", 2024-03-16T20:45:39.287 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCyBPZlxHcNHhAAfl+lxu1il8hS0+tJdg6OXA==", 2024-03-16T20:45:39.288 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.288 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow profile osd", 2024-03-16T20:45:39.288 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile osd", 2024-03-16T20:45:39.288 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-03-16T20:45:39.288 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.288 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.288 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.288 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.0", 2024-03-16T20:45:39.288 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCYBPZl3zQpMxAAhyRAHkPMirH6skmDMK55+A==", 2024-03-16T20:45:39.288 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.288 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow", 2024-03-16T20:45:39.288 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow r", 2024-03-16T20:45:39.288 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow rw", 2024-03-16T20:45:39.288 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow rwx" 2024-03-16T20:45:39.288 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.289 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.admin", 2024-03-16T20:45:39.289 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCYBPZlyEmTDhAA6UPaE2hykTdK5UB7uu7JJA==", 2024-03-16T20:45:39.289 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.289 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-03-16T20:45:39.289 INFO:teuthology.orchestra.run.smithi122.stdout: "mgr": "allow *", 2024-03-16T20:45:39.289 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow *", 2024-03-16T20:45:39.289 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-03-16T20:45:39.289 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.289 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.289 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.289 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.bootstrap-mds", 2024-03-16T20:45:39.289 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQC4BPZla5CwAhAAVo/IKVOBg9Cw2Fjbx4Cdcw==", 2024-03-16T20:45:39.289 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.290 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile bootstrap-mds" 2024-03-16T20:45:39.290 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.290 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.bootstrap-mgr", 2024-03-16T20:45:39.290 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQC4BPZlB6qwAhAAGESQt5JVMe6/2JSUgYESNQ==", 2024-03-16T20:45:39.290 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.290 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile bootstrap-mgr" 2024-03-16T20:45:39.290 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.290 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.290 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.290 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.bootstrap-osd", 2024-03-16T20:45:39.290 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQC4BPZltcCwAhAAp5fzHVzXGN7jsNUadcm1dw==", 2024-03-16T20:45:39.290 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.291 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile bootstrap-osd" 2024-03-16T20:45:39.291 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.291 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.bootstrap-rbd", 2024-03-16T20:45:39.291 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQC4BPZlLtiwAhAAMG1HebiyVtsj2rRLB43GnQ==", 2024-03-16T20:45:39.291 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.291 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile bootstrap-rbd" 2024-03-16T20:45:39.291 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.291 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.291 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.291 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-03-16T20:45:39.291 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQC4BPZlXe6wAhAAhbvEzzd8kFUR/A4LFm2qjA==", 2024-03-16T20:45:39.291 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.291 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-03-16T20:45:39.292 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.292 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "client.bootstrap-rgw", 2024-03-16T20:45:39.292 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQC4BPZlXgWxAhAAdzFDm45XE+v3OdUTEbrBWg==", 2024-03-16T20:45:39.292 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.292 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile bootstrap-rgw" 2024-03-16T20:45:39.292 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.292 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.292 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.292 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.x", 2024-03-16T20:45:39.292 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCYBPZlHLuGIhAA4dO9gHrVXmg0e3YvRS/5cg==", 2024-03-16T20:45:39.292 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.292 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-03-16T20:45:39.292 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-03-16T20:45:39.293 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-03-16T20:45:39.293 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.293 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:39.293 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:39.293 INFO:teuthology.orchestra.run.smithi122.stdout: "entity": "mgr.y", 2024-03-16T20:45:39.293 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "AQCYBPZlvgfHHxAAeorpgzjZnG6znj3MHGlTvA==", 2024-03-16T20:45:39.293 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": { 2024-03-16T20:45:39.293 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": "allow *", 2024-03-16T20:45:39.293 INFO:teuthology.orchestra.run.smithi122.stdout: "mon": "allow profile mgr", 2024-03-16T20:45:39.293 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "allow *" 2024-03-16T20:45:39.293 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.293 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:39.293 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-03-16T20:45:39.293 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-16T20:45:39.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.279+0000 7f356f1fd640 1 -- 172.21.15.122:0/1629164966 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3554066e40 msgr2=0x7f3554069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:39.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.279+0000 7f356f1fd640 1 --2- 172.21.15.122:0/1629164966 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3554066e40 0x7f3554069300 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3558000c10 tx=0x7f3558009040 comp rx=0 tx=0).stop 2024-03-16T20:45:39.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.279+0000 7f356f1fd640 1 -- 172.21.15.122:0/1629164966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3550003980 msgr2=0x7f35681431a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:39.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.279+0000 7f356f1fd640 1 --2- 172.21.15.122:0/1629164966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3550003980 0x7f35681431a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f355c01c5d0 tx=0x7f3550005800 comp rx=0 tx=0).stop 2024-03-16T20:45:39.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.279+0000 7f356f1fd640 1 -- 172.21.15.122:0/1629164966 shutdown_connections 2024-03-16T20:45:39.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.279+0000 7f356f1fd640 1 --2- 172.21.15.122:0/1629164966 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3554066e40 0x7f3554069300 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.279+0000 7f356f1fd640 1 --2- 172.21.15.122:0/1629164966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f35680a0c70 0x7f3568140c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.279+0000 7f356f1fd640 1 --2- 172.21.15.122:0/1629164966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3568142660 0x7f35680a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.279+0000 7f356f1fd640 1 --2- 172.21.15.122:0/1629164966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3550003980 0x7f35681431a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.279+0000 7f356f1fd640 1 -- 172.21.15.122:0/1629164966 >> 172.21.15.122:0/1629164966 conn(0x7f35680bca50 msgr2=0x7f3568146880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:39.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.279+0000 7f356f1fd640 1 -- 172.21.15.122:0/1629164966 shutdown_connections 2024-03-16T20:45:39.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.279+0000 7f356f1fd640 1 -- 172.21.15.122:0/1629164966 wait complete. 2024-03-16T20:45:39.300 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-03-16T20:45:39.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.419+0000 7f40a2f1b640 1 Processor -- start 2024-03-16T20:45:39.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.419+0000 7f40a2f1b640 1 -- start start 2024-03-16T20:45:39.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.419+0000 7f40a2f1b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f409c0b7d40 0x7f409c0b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.419+0000 7f40a2f1b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f409c148480 0x7f409c14f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.419+0000 7f40a2f1b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409c0b6740 con 0x7f409c0b7540 2024-03-16T20:45:39.424 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.419+0000 7f40a2f1b640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c0b23d0 con 0x7f409c148480 2024-03-16T20:45:39.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.419+0000 7f40a2f1b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409c0b7110 con 0x7f409c0b7d40 2024-03-16T20:45:39.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.419+0000 7f40a14b8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f409c148480 0x7f409c14f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.419+0000 7f40a14b8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f409c148480 0x7f409c14f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42122/0 (socket says 172.21.15.122:42122) 2024-03-16T20:45:39.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.419+0000 7f40a14b8640 1 -- 172.21.15.122:0/3499371916 learned_addr learned my addr 172.21.15.122:0/3499371916 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:39.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.419+0000 7f409bfff640 1 --2- 172.21.15.122:0/3499371916 >> v2:172.21.15.195:3300/0 conn(0x7f409c0b7d40 0x7f409c0b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.419+0000 7f40a14b8640 1 -- 172.21.15.122:0/3499371916 >> v2:172.21.15.195:3300/0 conn(0x7f409c0b7d40 msgr2=0x7f409c0b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:39.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.419+0000 7f40a14b8640 1 --2- 172.21.15.122:0/3499371916 >> v2:172.21.15.195:3300/0 conn(0x7f409c0b7d40 0x7f409c0b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.419+0000 7f40a14b8640 1 -- 172.21.15.122:0/3499371916 >> v1:172.21.15.195:6789/0 conn(0x7f409c0b7540 legacy=0x7f409c13ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:39.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.419+0000 7f40a14b8640 1 -- 172.21.15.122:0/3499371916 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c0b7920 con 0x7f409c148480 2024-03-16T20:45:39.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.419+0000 7f409bfff640 1 --2- 172.21.15.122:0/3499371916 >> v2:172.21.15.195:3300/0 conn(0x7f409c0b7d40 0x7f409c0b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:39.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a14b8640 1 --2- 172.21.15.122:0/3499371916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f409c148480 0x7f409c14f7b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f409c0b2d80 tx=0x7f409401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef4162759e605ea server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:39.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f409b7fe640 1 -- 172.21.15.122:0/3499371916 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40940a57b0 con 0x7f409c148480 2024-03-16T20:45:39.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f409b7fe640 1 -- 172.21.15.122:0/3499371916 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40940a5910 con 0x7f409c148480 2024-03-16T20:45:39.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f409b7fe640 1 -- 172.21.15.122:0/3499371916 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40940a5ca0 con 0x7f409c148480 2024-03-16T20:45:39.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a2f1b640 1 -- 172.21.15.122:0/3499371916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f409c148480 msgr2=0x7f409c14f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:39.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a2f1b640 1 --2- 172.21.15.122:0/3499371916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f409c148480 0x7f409c14f7b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f409c0b2d80 tx=0x7f409401c8f0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a2f1b640 1 -- 172.21.15.122:0/3499371916 shutdown_connections 2024-03-16T20:45:39.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a2f1b640 1 --2- 172.21.15.122:0/3499371916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f409c148480 0x7f409c14f7b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a2f1b640 1 --2- 172.21.15.122:0/3499371916 >> v2:172.21.15.195:3300/0 conn(0x7f409c0b7d40 0x7f409c0b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a2f1b640 1 -- 172.21.15.122:0/3499371916 >> 172.21.15.122:0/3499371916 conn(0x7f409c06fc00 msgr2=0x7f409c070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:39.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a2f1b640 1 -- 172.21.15.122:0/3499371916 shutdown_connections 2024-03-16T20:45:39.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a2f1b640 1 -- 172.21.15.122:0/3499371916 wait complete. 2024-03-16T20:45:39.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a2f1b640 1 Processor -- start 2024-03-16T20:45:39.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a2f1b640 1 -- start start 2024-03-16T20:45:39.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a2f1b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f409c0b7540 0x7f409c142fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a2f1b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f409c0b7d40 0x7f409c147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a2f1b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f409c148480 0x7f409c1e07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a2f1b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c13bb80 con 0x7f409c148480 2024-03-16T20:45:39.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a2f1b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409c144970 con 0x7f409c0b7d40 2024-03-16T20:45:39.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a2f1b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409c1466a0 con 0x7f409c0b7540 2024-03-16T20:45:39.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.423+0000 7f40a14b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f409c148480 0x7f409c1e07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40a14b8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f409c148480 0x7f409c1e07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42132/0 (socket says 172.21.15.122:42132) 2024-03-16T20:45:39.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40a14b8640 1 -- 172.21.15.122:0/1675452777 learned_addr learned my addr 172.21.15.122:0/1675452777 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:39.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f409bfff640 1 --2- 172.21.15.122:0/1675452777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f409c0b7d40 0x7f409c147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40a0cb7640 1 --2- 172.21.15.122:0/1675452777 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f409c0b7540 0x7f409c142fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40a14b8640 1 -- 172.21.15.122:0/1675452777 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f409c0b7540 msgr2=0x7f409c142fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40a14b8640 1 --2- 172.21.15.122:0/1675452777 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f409c0b7540 0x7f409c142fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40a14b8640 1 -- 172.21.15.122:0/1675452777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f409c0b7d40 msgr2=0x7f409c147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40a14b8640 1 --2- 172.21.15.122:0/1675452777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f409c0b7d40 0x7f409c147ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40a14b8640 1 -- 172.21.15.122:0/1675452777 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4094009b30 con 0x7f409c148480 2024-03-16T20:45:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40a0cb7640 1 --2- 172.21.15.122:0/1675452777 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f409c0b7540 0x7f409c142fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40a14b8640 1 --2- 172.21.15.122:0/1675452777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f409c148480 0x7f409c1e07c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f40940af040 tx=0x7f4094019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40997fa640 1 -- 172.21.15.122:0/1675452777 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40940b8670 con 0x7f409c148480 2024-03-16T20:45:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40997fa640 1 -- 172.21.15.122:0/1675452777 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40940b87d0 con 0x7f409c148480 2024-03-16T20:45:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40a2f1b640 1 -- 172.21.15.122:0/1675452777 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409c1472e0 con 0x7f409c148480 2024-03-16T20:45:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40997fa640 1 -- 172.21.15.122:0/1675452777 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40940b8b60 con 0x7f409c148480 2024-03-16T20:45:39.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40a2f1b640 1 -- 172.21.15.122:0/1675452777 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409c146070 con 0x7f409c148480 2024-03-16T20:45:39.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40a2f1b640 1 -- 172.21.15.122:0/1675452777 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f409c147790 con 0x7f409c148480 2024-03-16T20:45:39.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40997fa640 1 -- 172.21.15.122:0/1675452777 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f40940b8cc0 con 0x7f409c148480 2024-03-16T20:45:39.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40997fa640 1 --2- 172.21.15.122:0/1675452777 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4088066e70 0x7f4088069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40997fa640 1 -- 172.21.15.122:0/1675452777 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f409411ff00 con 0x7f409c148480 2024-03-16T20:45:39.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40a0cb7640 1 --2- 172.21.15.122:0/1675452777 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4088066e70 0x7f4088069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.427+0000 7f40a0cb7640 1 --2- 172.21.15.122:0/1675452777 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4088066e70 0x7f4088069330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f408c003e40 tx=0x7f408c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:39.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.431+0000 7f40997fa640 1 -- 172.21.15.122:0/1675452777 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f40940ed1f0 con 0x7f409c148480 2024-03-16T20:45:39.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.663+0000 7f40a2f1b640 1 -- 172.21.15.122:0/1675452777 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f409c0b2d90 con 0x7f409c148480 2024-03-16T20:45:39.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.663+0000 7f40997fa640 1 -- 172.21.15.122:0/1675452777 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f40940f08b0 con 0x7f409c148480 2024-03-16T20:45:39.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.667+0000 7f40a2f1b640 1 -- 172.21.15.122:0/1675452777 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4088066e70 msgr2=0x7f4088069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:39.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.667+0000 7f40a2f1b640 1 --2- 172.21.15.122:0/1675452777 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4088066e70 0x7f4088069330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f408c003e40 tx=0x7f408c009040 comp rx=0 tx=0).stop 2024-03-16T20:45:39.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.667+0000 7f40a2f1b640 1 -- 172.21.15.122:0/1675452777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f409c148480 msgr2=0x7f409c1e07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:39.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.667+0000 7f40a2f1b640 1 --2- 172.21.15.122:0/1675452777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f409c148480 0x7f409c1e07c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f40940af040 tx=0x7f4094019690 comp rx=0 tx=0).stop 2024-03-16T20:45:39.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.667+0000 7f40a2f1b640 1 -- 172.21.15.122:0/1675452777 shutdown_connections 2024-03-16T20:45:39.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.667+0000 7f40a2f1b640 1 --2- 172.21.15.122:0/1675452777 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4088066e70 0x7f4088069330 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.667+0000 7f40a2f1b640 1 --2- 172.21.15.122:0/1675452777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f409c148480 0x7f409c1e07c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.667+0000 7f40a2f1b640 1 --2- 172.21.15.122:0/1675452777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f409c0b7d40 0x7f409c147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.667+0000 7f40a2f1b640 1 --2- 172.21.15.122:0/1675452777 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f409c0b7540 0x7f409c142fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.667+0000 7f40a2f1b640 1 -- 172.21.15.122:0/1675452777 >> 172.21.15.122:0/1675452777 conn(0x7f409c06fc00 msgr2=0x7f409c0b3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:39.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.667+0000 7f40a2f1b640 1 -- 172.21.15.122:0/1675452777 shutdown_connections 2024-03-16T20:45:39.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.667+0000 7f40a2f1b640 1 -- 172.21.15.122:0/1675452777 wait complete. 2024-03-16T20:45:39.689 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-03-16T20:45:39.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.803+0000 7fb63572d640 1 Processor -- start 2024-03-16T20:45:39.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.803+0000 7fb63572d640 1 -- start start 2024-03-16T20:45:39.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.803+0000 7fb63572d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb6301429f0 0x7fb6300b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.803+0000 7fb63572d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6300b3fa0 con 0x7fb63009ae30 2024-03-16T20:45:39.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.803+0000 7fb63572d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb63009b210 con 0x7fb6300b82b0 2024-03-16T20:45:39.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.803+0000 7fb63572d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6300b50a0 con 0x7fb6301429f0 2024-03-16T20:45:39.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.803+0000 7fb62effd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fb63009ae30 0x7fb6300b7ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39446/0 (socket says 172.21.15.122:39446) 2024-03-16T20:45:39.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.803+0000 7fb62effd640 1 -- 172.21.15.122:0/1313413937 learned_addr learned my addr 172.21.15.122:0/1313413937 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:39.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.803+0000 7fb62f7fe640 1 --2- 172.21.15.122:0/1313413937 >> v2:172.21.15.122:3300/0 conn(0x7fb6301429f0 0x7fb6300b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.803+0000 7fb62dffb640 1 -- 172.21.15.122:0/1313413937 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136750583 0 0) 0x7fb6300b3fa0 con 0x7fb63009ae30 2024-03-16T20:45:39.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.803+0000 7fb62dffb640 1 -- 172.21.15.122:0/1313413937 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb61c003530 con 0x7fb63009ae30 2024-03-16T20:45:39.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.803+0000 7fb62f7fe640 1 -- 172.21.15.122:0/1313413937 >> v1:172.21.15.195:6789/0 conn(0x7fb6300b82b0 legacy=0x7fb6301422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:39.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.803+0000 7fb62f7fe640 1 -- 172.21.15.122:0/1313413937 >> v1:172.21.15.122:6789/0 conn(0x7fb63009ae30 legacy=0x7fb6300b7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:39.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.803+0000 7fb62f7fe640 1 -- 172.21.15.122:0/1313413937 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6300b4ae0 con 0x7fb6301429f0 2024-03-16T20:45:39.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb62f7fe640 1 --2- 172.21.15.122:0/1313413937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6301429f0 0x7fb6300b4430 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb6200097f0 tx=0x7fb62001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1ab472aa122272 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:39.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb62dffb640 1 -- 172.21.15.122:0/1313413937 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6200ab680 con 0x7fb6301429f0 2024-03-16T20:45:39.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb62dffb640 1 -- 172.21.15.122:0/1313413937 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6200ab7e0 con 0x7fb6301429f0 2024-03-16T20:45:39.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb62dffb640 1 -- 172.21.15.122:0/1313413937 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6200abb70 con 0x7fb6301429f0 2024-03-16T20:45:39.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb63572d640 1 -- 172.21.15.122:0/1313413937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6301429f0 msgr2=0x7fb6300b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:39.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb63572d640 1 --2- 172.21.15.122:0/1313413937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6301429f0 0x7fb6300b4430 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb6200097f0 tx=0x7fb62001c460 comp rx=0 tx=0).stop 2024-03-16T20:45:39.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb63572d640 1 -- 172.21.15.122:0/1313413937 shutdown_connections 2024-03-16T20:45:39.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb63572d640 1 --2- 172.21.15.122:0/1313413937 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6301429f0 0x7fb6300b4430 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb63572d640 1 -- 172.21.15.122:0/1313413937 >> 172.21.15.122:0/1313413937 conn(0x7fb6300b8d10 msgr2=0x7fb63009a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:39.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb63572d640 1 -- 172.21.15.122:0/1313413937 shutdown_connections 2024-03-16T20:45:39.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb63572d640 1 -- 172.21.15.122:0/1313413937 wait complete. 2024-03-16T20:45:39.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb63572d640 1 Processor -- start 2024-03-16T20:45:39.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb63572d640 1 -- start start 2024-03-16T20:45:39.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb63572d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63009ae30 0x7fb63013a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb63572d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6300b82b0 0x7fb63013abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb63572d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6301429f0 0x7fb6300720d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb63572d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63013d110 con 0x7fb6300b82b0 2024-03-16T20:45:39.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb63572d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63013bd10 con 0x7fb6301429f0 2024-03-16T20:45:39.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb63572d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63013bff0 con 0x7fb63009ae30 2024-03-16T20:45:39.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb62e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6300b82b0 0x7fb63013abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb62e7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6300b82b0 0x7fb63013abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42148/0 (socket says 172.21.15.122:42148) 2024-03-16T20:45:39.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb62e7fc640 1 -- 172.21.15.122:0/3668443761 learned_addr learned my addr 172.21.15.122:0/3668443761 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb62effd640 1 --2- 172.21.15.122:0/3668443761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63009ae30 0x7fb63013a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb62e7fc640 1 -- 172.21.15.122:0/3668443761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63009ae30 msgr2=0x7fb63013a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb62e7fc640 1 --2- 172.21.15.122:0/3668443761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63009ae30 0x7fb63013a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb62e7fc640 1 -- 172.21.15.122:0/3668443761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6301429f0 msgr2=0x7fb6300720d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb62e7fc640 1 --2- 172.21.15.122:0/3668443761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6301429f0 0x7fb6300720d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb62e7fc640 1 -- 172.21.15.122:0/3668443761 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb620009420 con 0x7fb6300b82b0 2024-03-16T20:45:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb62effd640 1 --2- 172.21.15.122:0/3668443761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63009ae30 0x7fb63013a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb62e7fc640 1 --2- 172.21.15.122:0/3668443761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6300b82b0 0x7fb63013abf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb61800c400 tx=0x7fb61800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb6277fe640 1 -- 172.21.15.122:0/3668443761 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618016070 con 0x7fb6300b82b0 2024-03-16T20:45:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb6277fe640 1 -- 172.21.15.122:0/3668443761 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb618007020 con 0x7fb6300b82b0 2024-03-16T20:45:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb63572d640 1 -- 172.21.15.122:0/3668443761 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb63013cac0 con 0x7fb6300b82b0 2024-03-16T20:45:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb6277fe640 1 -- 172.21.15.122:0/3668443761 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb618007410 con 0x7fb6300b82b0 2024-03-16T20:45:39.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.807+0000 7fb63572d640 1 -- 172.21.15.122:0/3668443761 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb63013b7e0 con 0x7fb6300b82b0 2024-03-16T20:45:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.811+0000 7fb63572d640 1 -- 172.21.15.122:0/3668443761 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb630074bb0 con 0x7fb6300b82b0 2024-03-16T20:45:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.811+0000 7fb6277fe640 1 -- 172.21.15.122:0/3668443761 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fb618007570 con 0x7fb6300b82b0 2024-03-16T20:45:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.811+0000 7fb6277fe640 1 --2- 172.21.15.122:0/3668443761 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb610066e70 0x7fb610069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.811+0000 7fb6277fe640 1 -- 172.21.15.122:0/3668443761 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb618083ed0 con 0x7fb6300b82b0 2024-03-16T20:45:39.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.811+0000 7fb62effd640 1 --2- 172.21.15.122:0/3668443761 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb610066e70 0x7fb610069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:39.817 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.811+0000 7fb62effd640 1 --2- 172.21.15.122:0/3668443761 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb610066e70 0x7fb610069330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb6280012f0 tx=0x7fb628009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:39.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:39.815+0000 7fb6277fe640 1 -- 172.21.15.122:0/3668443761 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fb6180510a0 con 0x7fb6300b82b0 2024-03-16T20:45:40.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.031+0000 7fb63572d640 1 -- 172.21.15.122:0/3668443761 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fb630071e30 con 0x7fb6300b82b0 2024-03-16T20:45:40.038 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.035+0000 7fb6277fe640 1 -- 172.21.15.122:0/3668443761 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb618054760 con 0x7fb6300b82b0 2024-03-16T20:45:40.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.035+0000 7fb63572d640 1 -- 172.21.15.122:0/3668443761 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb610066e70 msgr2=0x7fb610069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:40.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.035+0000 7fb63572d640 1 --2- 172.21.15.122:0/3668443761 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb610066e70 0x7fb610069330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb6280012f0 tx=0x7fb628009040 comp rx=0 tx=0).stop 2024-03-16T20:45:40.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.035+0000 7fb63572d640 1 -- 172.21.15.122:0/3668443761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6300b82b0 msgr2=0x7fb63013abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:40.041 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.035+0000 7fb63572d640 1 --2- 172.21.15.122:0/3668443761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6300b82b0 0x7fb63013abf0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb61800c400 tx=0x7fb61800c8c0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.035+0000 7fb63572d640 1 -- 172.21.15.122:0/3668443761 shutdown_connections 2024-03-16T20:45:40.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.035+0000 7fb63572d640 1 --2- 172.21.15.122:0/3668443761 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb610066e70 0x7fb610069330 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.035+0000 7fb63572d640 1 --2- 172.21.15.122:0/3668443761 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb6301429f0 0x7fb6300720d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.035+0000 7fb63572d640 1 --2- 172.21.15.122:0/3668443761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6300b82b0 0x7fb63013abf0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.035+0000 7fb63572d640 1 --2- 172.21.15.122:0/3668443761 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb63009ae30 0x7fb63013a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.035+0000 7fb63572d640 1 -- 172.21.15.122:0/3668443761 >> 172.21.15.122:0/3668443761 conn(0x7fb6300b8d10 msgr2=0x7fb630099be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:40.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.035+0000 7fb63572d640 1 -- 172.21.15.122:0/3668443761 shutdown_connections 2024-03-16T20:45:40.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.035+0000 7fb63572d640 1 -- 172.21.15.122:0/3668443761 wait complete. 2024-03-16T20:45:40.056 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-03-16T20:45:40.166 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa83e78d640 1 Processor -- start 2024-03-16T20:45:40.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa83e78d640 1 -- start start 2024-03-16T20:45:40.167 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa83e78d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa83809ae30 0x7fa8380b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa83e78d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa8380b7ba0 0x7fa8380b8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa83e78d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8380b3fa0 con 0x7fa8381422e0 2024-03-16T20:45:40.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa83e78d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8380b50a0 con 0x7fa8380b7ba0 2024-03-16T20:45:40.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa83e78d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8380b8590 con 0x7fa83809ae30 2024-03-16T20:45:40.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa8377fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa8380b7ba0 0x7fa8380b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa8377fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fa8380b7ba0 0x7fa8380b8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42162/0 (socket says 172.21.15.122:42162) 2024-03-16T20:45:40.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa8377fe640 1 -- 172.21.15.122:0/2495577889 learned_addr learned my addr 172.21.15.122:0/2495577889 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:40.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa8377fe640 1 -- 172.21.15.122:0/2495577889 >> v2:172.21.15.195:3300/0 conn(0x7fa83809ae30 msgr2=0x7fa8380b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:40.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa8377fe640 1 --2- 172.21.15.122:0/2495577889 >> v2:172.21.15.195:3300/0 conn(0x7fa83809ae30 0x7fa8380b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa8377fe640 1 -- 172.21.15.122:0/2495577889 >> v1:172.21.15.195:6789/0 conn(0x7fa8381422e0 legacy=0x7fa838142770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:40.168 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa8377fe640 1 -- 172.21.15.122:0/2495577889 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83809b210 con 0x7fa8380b7ba0 2024-03-16T20:45:40.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa8377fe640 1 --2- 172.21.15.122:0/2495577889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8380b7ba0 0x7fa8380b8050 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa83809e0e0 tx=0x7fa82801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aedc0aa1a50e77bc server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:40.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa836ffd640 1 -- 172.21.15.122:0/2495577889 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8280aa640 con 0x7fa8380b7ba0 2024-03-16T20:45:40.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa836ffd640 1 -- 172.21.15.122:0/2495577889 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8280aa7a0 con 0x7fa8380b7ba0 2024-03-16T20:45:40.170 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.163+0000 7fa836ffd640 1 -- 172.21.15.122:0/2495577889 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8280aab30 con 0x7fa8380b7ba0 2024-03-16T20:45:40.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 -- 172.21.15.122:0/2495577889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8380b7ba0 msgr2=0x7fa8380b8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:40.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 --2- 172.21.15.122:0/2495577889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8380b7ba0 0x7fa8380b8050 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa83809e0e0 tx=0x7fa82801c4e0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 -- 172.21.15.122:0/2495577889 shutdown_connections 2024-03-16T20:45:40.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 --2- 172.21.15.122:0/2495577889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8380b7ba0 0x7fa8380b8050 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 --2- 172.21.15.122:0/2495577889 >> v2:172.21.15.195:3300/0 conn(0x7fa83809ae30 0x7fa8380b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 -- 172.21.15.122:0/2495577889 >> 172.21.15.122:0/2495577889 conn(0x7fa8380b8d10 msgr2=0x7fa83809a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:40.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 -- 172.21.15.122:0/2495577889 shutdown_connections 2024-03-16T20:45:40.171 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 -- 172.21.15.122:0/2495577889 wait complete. 2024-03-16T20:45:40.173 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 Processor -- start 2024-03-16T20:45:40.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 -- start start 2024-03-16T20:45:40.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa83809ae30 0x7fa8381ddcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8380b7ba0 0x7fa838144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8381422e0 0x7fa8381e0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8381dd300 con 0x7fa8380b7ba0 2024-03-16T20:45:40.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8381de230 con 0x7fa8381422e0 2024-03-16T20:45:40.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8380afc40 con 0x7fa83809ae30 2024-03-16T20:45:40.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa8377fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8380b7ba0 0x7fa838144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa8377fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8380b7ba0 0x7fa838144af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42166/0 (socket says 172.21.15.122:42166) 2024-03-16T20:45:40.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa8377fe640 1 -- 172.21.15.122:0/2844541048 learned_addr learned my addr 172.21.15.122:0/2844541048 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:40.174 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa8377fe640 1 -- 172.21.15.122:0/2844541048 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa83809ae30 msgr2=0x7fa8381ddcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:40.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa8377fe640 1 --2- 172.21.15.122:0/2844541048 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa83809ae30 0x7fa8381ddcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa8377fe640 1 -- 172.21.15.122:0/2844541048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8381422e0 msgr2=0x7fa8381e0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:40.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83cd2a640 1 --2- 172.21.15.122:0/2844541048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8381422e0 0x7fa8381e0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa8377fe640 1 --2- 172.21.15.122:0/2844541048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8381422e0 0x7fa8381e0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa8377fe640 1 -- 172.21.15.122:0/2844541048 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8280094d0 con 0x7fa8380b7ba0 2024-03-16T20:45:40.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa8377fe640 1 --2- 172.21.15.122:0/2844541048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8380b7ba0 0x7fa838144af0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa82801a110 tx=0x7fa82801ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:40.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa834ff9640 1 -- 172.21.15.122:0/2844541048 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8280b36b0 con 0x7fa8380b7ba0 2024-03-16T20:45:40.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa834ff9640 1 -- 172.21.15.122:0/2844541048 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8280b3810 con 0x7fa8380b7ba0 2024-03-16T20:45:40.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 -- 172.21.15.122:0/2844541048 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83813d020 con 0x7fa8380b7ba0 2024-03-16T20:45:40.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa834ff9640 1 -- 172.21.15.122:0/2844541048 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8280b3ba0 con 0x7fa8380b7ba0 2024-03-16T20:45:40.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 -- 172.21.15.122:0/2844541048 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8381dd7d0 con 0x7fa8380b7ba0 2024-03-16T20:45:40.175 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.167+0000 7fa83e78d640 1 -- 172.21.15.122:0/2844541048 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa83809a670 con 0x7fa8380b7ba0 2024-03-16T20:45:40.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.171+0000 7fa834ff9640 1 -- 172.21.15.122:0/2844541048 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fa8280a8050 con 0x7fa8380b7ba0 2024-03-16T20:45:40.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.171+0000 7fa834ff9640 1 --2- 172.21.15.122:0/2844541048 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa824066e70 0x7fa824069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.171+0000 7fa834ff9640 1 -- 172.21.15.122:0/2844541048 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa82811ed80 con 0x7fa8380b7ba0 2024-03-16T20:45:40.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.171+0000 7fa837fff640 1 --2- 172.21.15.122:0/2844541048 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa824066e70 0x7fa824069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.176 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.171+0000 7fa837fff640 1 --2- 172.21.15.122:0/2844541048 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa824066e70 0x7fa824069330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa83813cc20 tx=0x7fa82c006ca0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:40.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.175+0000 7fa834ff9640 1 -- 172.21.15.122:0/2844541048 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa8280ebf50 con 0x7fa8380b7ba0 2024-03-16T20:45:40.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.383+0000 7fa83e78d640 1 -- 172.21.15.122:0/2844541048 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fa83813dd80 con 0x7fa8380b7ba0 2024-03-16T20:45:40.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.387+0000 7fa834ff9640 1 -- 172.21.15.122:0/2844541048 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7fa8280ef610 con 0x7fa8380b7ba0 2024-03-16T20:45:40.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.391+0000 7fa83e78d640 1 -- 172.21.15.122:0/2844541048 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa824066e70 msgr2=0x7fa824069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:40.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.391+0000 7fa83e78d640 1 --2- 172.21.15.122:0/2844541048 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa824066e70 0x7fa824069330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa83813cc20 tx=0x7fa82c006ca0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.391+0000 7fa83e78d640 1 -- 172.21.15.122:0/2844541048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8380b7ba0 msgr2=0x7fa838144af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:40.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.391+0000 7fa83e78d640 1 --2- 172.21.15.122:0/2844541048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8380b7ba0 0x7fa838144af0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa82801a110 tx=0x7fa82801ad20 comp rx=0 tx=0).stop 2024-03-16T20:45:40.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.391+0000 7fa83e78d640 1 -- 172.21.15.122:0/2844541048 shutdown_connections 2024-03-16T20:45:40.395 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.391+0000 7fa83e78d640 1 --2- 172.21.15.122:0/2844541048 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa824066e70 0x7fa824069330 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.391+0000 7fa83e78d640 1 --2- 172.21.15.122:0/2844541048 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa8381422e0 0x7fa8381e0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.391+0000 7fa83e78d640 1 --2- 172.21.15.122:0/2844541048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8380b7ba0 0x7fa838144af0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.391+0000 7fa83e78d640 1 --2- 172.21.15.122:0/2844541048 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa83809ae30 0x7fa8381ddcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.391+0000 7fa83e78d640 1 -- 172.21.15.122:0/2844541048 >> 172.21.15.122:0/2844541048 conn(0x7fa8380b8d10 msgr2=0x7fa83806d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:40.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.391+0000 7fa83e78d640 1 -- 172.21.15.122:0/2844541048 shutdown_connections 2024-03-16T20:45:40.396 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.391+0000 7fa83e78d640 1 -- 172.21.15.122:0/2844541048 wait complete. 2024-03-16T20:45:40.410 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-03-16T20:45:40.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc5529a6640 1 Processor -- start 2024-03-16T20:45:40.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc5529a6640 1 -- start start 2024-03-16T20:45:40.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc5529a6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc54c06e030 0x7fc54c144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc5529a6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc54c13f4a0 0x7fc54c14f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc5529a6640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54c09a130 con 0x7fc54c145510 2024-03-16T20:45:40.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc5529a6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54c06e410 con 0x7fc54c06e030 2024-03-16T20:45:40.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc5529a6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c099de0 con 0x7fc54c13f4a0 2024-03-16T20:45:40.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54b7fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fc54c145510 0x7fc54c13ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39454/0 (socket says 172.21.15.122:39454) 2024-03-16T20:45:40.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54b7fe640 1 -- 172.21.15.122:0/1487879264 learned_addr learned my addr 172.21.15.122:0/1487879264 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:40.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54affd640 1 -- 172.21.15.122:0/1487879264 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919295739 0 0) 0x7fc54c09a130 con 0x7fc54c145510 2024-03-16T20:45:40.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54affd640 1 -- 172.21.15.122:0/1487879264 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc538003530 con 0x7fc54c145510 2024-03-16T20:45:40.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54bfff640 1 --2- 172.21.15.122:0/1487879264 >> v2:172.21.15.195:3300/0 conn(0x7fc54c06e030 0x7fc54c144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc550f43640 1 --2- 172.21.15.122:0/1487879264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc54c13f4a0 0x7fc54c14f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54affd640 1 -- 172.21.15.122:0/1487879264 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 246744651 0 0) 0x7fc538003530 con 0x7fc54c145510 2024-03-16T20:45:40.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54affd640 1 -- 172.21.15.122:0/1487879264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc54c13f4a0 msgr2=0x7fc54c14f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:40.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54affd640 1 --2- 172.21.15.122:0/1487879264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc54c13f4a0 0x7fc54c14f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54affd640 1 -- 172.21.15.122:0/1487879264 >> v2:172.21.15.195:3300/0 conn(0x7fc54c06e030 msgr2=0x7fc54c144fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:40.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54affd640 1 --2- 172.21.15.122:0/1487879264 >> v2:172.21.15.195:3300/0 conn(0x7fc54c06e030 0x7fc54c144fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54affd640 1 -- 172.21.15.122:0/1487879264 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c06c620 con 0x7fc54c145510 2024-03-16T20:45:40.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54affd640 1 -- 172.21.15.122:0/1487879264 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7fc534001680 con 0x7fc54c145510 2024-03-16T20:45:40.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54affd640 1 -- 172.21.15.122:0/1487879264 >> v1:172.21.15.122:6789/0 conn(0x7fc54c145510 legacy=0x7fc54c13ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:40.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54affd640 1 --2- 172.21.15.122:0/1487879264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc538003980 0x7fc538003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54affd640 1 --2- 172.21.15.122:0/1487879264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc54c13f4a0 0x7fc5380047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54affd640 1 --2- 172.21.15.122:0/1487879264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc538004d10 0x7fc538005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54affd640 1 -- 172.21.15.122:0/1487879264 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c06e410 con 0x7fc538003980 2024-03-16T20:45:40.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54affd640 1 -- 172.21.15.122:0/1487879264 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c099de0 con 0x7fc538004d10 2024-03-16T20:45:40.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54affd640 1 -- 172.21.15.122:0/1487879264 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53800ba40 con 0x7fc54c13f4a0 2024-03-16T20:45:40.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54b7fe640 1 --2- 172.21.15.122:0/1487879264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc54c13f4a0 0x7fc5380047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54bfff640 1 --2- 172.21.15.122:0/1487879264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc538003980 0x7fc538003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc550f43640 1 --2- 172.21.15.122:0/1487879264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc538004d10 0x7fc538005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54bfff640 1 -- 172.21.15.122:0/1487879264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc54c13f4a0 msgr2=0x7fc5380047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:40.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54bfff640 1 --2- 172.21.15.122:0/1487879264 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc54c13f4a0 0x7fc5380047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54bfff640 1 -- 172.21.15.122:0/1487879264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc538004d10 msgr2=0x7fc538005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:40.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54bfff640 1 --2- 172.21.15.122:0/1487879264 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc538004d10 0x7fc538005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.515+0000 7fc54bfff640 1 -- 172.21.15.122:0/1487879264 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc538003530 con 0x7fc538003980 2024-03-16T20:45:40.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc54bfff640 1 --2- 172.21.15.122:0/1487879264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc538003980 0x7fc538003da0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc53c009fc0 tx=0x7fc53c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff0630a7e2bc56b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:40.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc54affd640 1 -- 172.21.15.122:0/1487879264 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c0a5bc0 con 0x7fc538003980 2024-03-16T20:45:40.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc54affd640 1 -- 172.21.15.122:0/1487879264 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53c00f040 con 0x7fc538003980 2024-03-16T20:45:40.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc54affd640 1 -- 172.21.15.122:0/1487879264 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c0ad440 con 0x7fc538003980 2024-03-16T20:45:40.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc5529a6640 1 -- 172.21.15.122:0/1487879264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc538003980 msgr2=0x7fc538003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:40.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc5529a6640 1 --2- 172.21.15.122:0/1487879264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc538003980 0x7fc538003da0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc53c009fc0 tx=0x7fc53c01ccc0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc550f43640 1 -- 172.21.15.122:0/1487879264 reap_dead start 2024-03-16T20:45:40.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc5529a6640 1 -- 172.21.15.122:0/1487879264 shutdown_connections 2024-03-16T20:45:40.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc5529a6640 1 -- 172.21.15.122:0/1487879264 >> 172.21.15.122:0/1487879264 conn(0x7fc54c06d2b0 msgr2=0x7fc54c06d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:40.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc5529a6640 1 -- 172.21.15.122:0/1487879264 shutdown_connections 2024-03-16T20:45:40.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc5529a6640 1 -- 172.21.15.122:0/1487879264 wait complete. 2024-03-16T20:45:40.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc5529a6640 1 Processor -- start 2024-03-16T20:45:40.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc5529a6640 1 -- start start 2024-03-16T20:45:40.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc5529a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc538003980 0x7fc54c147360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc5529a6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc54c1420e0 0x7fc54c140720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc5529a6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc54c1411b0 0x7fc54c13d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc5529a6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c1370d0 con 0x7fc538003980 2024-03-16T20:45:40.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc5529a6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c13f9f0 con 0x7fc54c1411b0 2024-03-16T20:45:40.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc5529a6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c140290 con 0x7fc54c1420e0 2024-03-16T20:45:40.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc54bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc538003980 0x7fc54c147360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc54bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc538003980 0x7fc54c147360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42178/0 (socket says 172.21.15.122:42178) 2024-03-16T20:45:40.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc54bfff640 1 -- 172.21.15.122:0/2996196966 learned_addr learned my addr 172.21.15.122:0/2996196966 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:40.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc54b7fe640 1 --2- 172.21.15.122:0/2996196966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc54c1420e0 0x7fc54c140720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc550f43640 1 --2- 172.21.15.122:0/2996196966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc54c1411b0 0x7fc54c13d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc54bfff640 1 -- 172.21.15.122:0/2996196966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc54c1420e0 msgr2=0x7fc54c140720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:40.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc54bfff640 1 --2- 172.21.15.122:0/2996196966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc54c1420e0 0x7fc54c140720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc54bfff640 1 -- 172.21.15.122:0/2996196966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc54c1411b0 msgr2=0x7fc54c13d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:40.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc54bfff640 1 --2- 172.21.15.122:0/2996196966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc54c1411b0 0x7fc54c13d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.519+0000 7fc54bfff640 1 -- 172.21.15.122:0/2996196966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c142610 con 0x7fc538003980 2024-03-16T20:45:40.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.523+0000 7fc54bfff640 1 --2- 172.21.15.122:0/2996196966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc538003980 0x7fc54c147360 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc53c01c730 tx=0x7fc53c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:40.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.523+0000 7fc548ff9640 1 -- 172.21.15.122:0/2996196966 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c0ad700 con 0x7fc538003980 2024-03-16T20:45:40.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.523+0000 7fc548ff9640 1 -- 172.21.15.122:0/2996196966 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53c0ad860 con 0x7fc538003980 2024-03-16T20:45:40.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.523+0000 7fc548ff9640 1 -- 172.21.15.122:0/2996196966 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c0adbf0 con 0x7fc538003980 2024-03-16T20:45:40.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.523+0000 7fc5529a6640 1 -- 172.21.15.122:0/2996196966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c13fc40 con 0x7fc538003980 2024-03-16T20:45:40.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.523+0000 7fc5529a6640 1 -- 172.21.15.122:0/2996196966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c072530 con 0x7fc538003980 2024-03-16T20:45:40.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.523+0000 7fc5529a6640 1 -- 172.21.15.122:0/2996196966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc54c074bb0 con 0x7fc538003980 2024-03-16T20:45:40.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.523+0000 7fc548ff9640 1 -- 172.21.15.122:0/2996196966 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fc53c0b4040 con 0x7fc538003980 2024-03-16T20:45:40.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.523+0000 7fc548ff9640 1 --2- 172.21.15.122:0/2996196966 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc52c066e40 0x7fc52c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.523+0000 7fc548ff9640 1 -- 172.21.15.122:0/2996196966 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc53c11efb0 con 0x7fc538003980 2024-03-16T20:45:40.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.523+0000 7fc54b7fe640 1 --2- 172.21.15.122:0/2996196966 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc52c066e40 0x7fc52c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.523+0000 7fc54b7fe640 1 --2- 172.21.15.122:0/2996196966 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc52c066e40 0x7fc52c069300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc534001060 tx=0x7fc5340011a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:40.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.527+0000 7fc548ff9640 1 -- 172.21.15.122:0/2996196966 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc53c0ec0e0 con 0x7fc538003980 2024-03-16T20:45:40.730 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.727+0000 7fc5529a6640 1 -- 172.21.15.122:0/2996196966 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fc54c1493d0 con 0x7fc538003980 2024-03-16T20:45:40.732 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.727+0000 7fc548ff9640 1 -- 172.21.15.122:0/2996196966 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fc53c0ef7a0 con 0x7fc538003980 2024-03-16T20:45:40.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.731+0000 7fc5529a6640 1 -- 172.21.15.122:0/2996196966 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc52c066e40 msgr2=0x7fc52c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:40.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.731+0000 7fc5529a6640 1 --2- 172.21.15.122:0/2996196966 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc52c066e40 0x7fc52c069300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc534001060 tx=0x7fc5340011a0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.731+0000 7fc5529a6640 1 -- 172.21.15.122:0/2996196966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc538003980 msgr2=0x7fc54c147360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:40.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.731+0000 7fc5529a6640 1 --2- 172.21.15.122:0/2996196966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc538003980 0x7fc54c147360 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc53c01c730 tx=0x7fc53c009750 comp rx=0 tx=0).stop 2024-03-16T20:45:40.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.731+0000 7fc5529a6640 1 -- 172.21.15.122:0/2996196966 shutdown_connections 2024-03-16T20:45:40.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.731+0000 7fc5529a6640 1 --2- 172.21.15.122:0/2996196966 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc52c066e40 0x7fc52c069300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.731+0000 7fc5529a6640 1 --2- 172.21.15.122:0/2996196966 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc54c1411b0 0x7fc54c13d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.731+0000 7fc5529a6640 1 --2- 172.21.15.122:0/2996196966 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc54c1420e0 0x7fc54c140720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.731+0000 7fc5529a6640 1 --2- 172.21.15.122:0/2996196966 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc538003980 0x7fc54c147360 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.731+0000 7fc5529a6640 1 -- 172.21.15.122:0/2996196966 >> 172.21.15.122:0/2996196966 conn(0x7fc54c06d2b0 msgr2=0x7fc54c0b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:40.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.731+0000 7fc5529a6640 1 -- 172.21.15.122:0/2996196966 shutdown_connections 2024-03-16T20:45:40.736 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.731+0000 7fc5529a6640 1 -- 172.21.15.122:0/2996196966 wait complete. 2024-03-16T20:45:40.750 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-03-16T20:45:40.862 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2181309640 1 Processor -- start 2024-03-16T20:45:40.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2181309640 1 -- start start 2024-03-16T20:45:40.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2181309640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f217c148f40 0x7f217c0b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2181309640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f217c14d700 0x7f217c14faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2181309640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217c149320 con 0x7f217c1483a0 2024-03-16T20:45:40.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2181309640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f217c0b3d50 con 0x7f217c14d700 2024-03-16T20:45:40.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2181309640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0b3140 con 0x7f217c148f40 2024-03-16T20:45:40.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f217a7fc640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f217c1483a0 0x7f217c0b6260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39468/0 (socket says 172.21.15.122:39468) 2024-03-16T20:45:40.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f217a7fc640 1 -- 172.21.15.122:0/794081594 learned_addr learned my addr 172.21.15.122:0/794081594 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:40.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f217b7fe640 1 --2- 172.21.15.122:0/794081594 >> v2:172.21.15.195:3300/0 conn(0x7f217c14d700 0x7f217c14faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f217affd640 1 --2- 172.21.15.122:0/794081594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f217c148f40 0x7f217c0b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2179ffb640 1 -- 172.21.15.122:0/794081594 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1327525694 0 0) 0x7f217c149320 con 0x7f217c1483a0 2024-03-16T20:45:40.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2179ffb640 1 -- 172.21.15.122:0/794081594 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2168003530 con 0x7f217c1483a0 2024-03-16T20:45:40.865 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2179ffb640 1 -- 172.21.15.122:0/794081594 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 494471409 0 0) 0x7f2168003530 con 0x7f217c1483a0 2024-03-16T20:45:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2179ffb640 1 -- 172.21.15.122:0/794081594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f217c148f40 msgr2=0x7f217c0b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2179ffb640 1 --2- 172.21.15.122:0/794081594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f217c148f40 0x7f217c0b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2179ffb640 1 -- 172.21.15.122:0/794081594 >> v2:172.21.15.195:3300/0 conn(0x7f217c14d700 msgr2=0x7f217c14faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2179ffb640 1 --2- 172.21.15.122:0/794081594 >> v2:172.21.15.195:3300/0 conn(0x7f217c14d700 0x7f217c14faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2179ffb640 1 -- 172.21.15.122:0/794081594 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c0b6970 con 0x7f217c1483a0 2024-03-16T20:45:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2179ffb640 1 -- 172.21.15.122:0/794081594 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f2164001680 con 0x7f217c1483a0 2024-03-16T20:45:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2179ffb640 1 -- 172.21.15.122:0/794081594 >> v1:172.21.15.122:6789/0 conn(0x7f217c1483a0 legacy=0x7f217c0b6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2179ffb640 1 --2- 172.21.15.122:0/794081594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2168003980 0x7f2168003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2179ffb640 1 --2- 172.21.15.122:0/794081594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21680047d0 0x7f2168004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2179ffb640 1 --2- 172.21.15.122:0/794081594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f217c148f40 0x7f2168005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2179ffb640 1 -- 172.21.15.122:0/794081594 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0b3d50 con 0x7f2168003980 2024-03-16T20:45:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2179ffb640 1 -- 172.21.15.122:0/794081594 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c0b3140 con 0x7f21680047d0 2024-03-16T20:45:40.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f2179ffb640 1 -- 172.21.15.122:0/794081594 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216800b9c0 con 0x7f217c148f40 2024-03-16T20:45:40.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f217affd640 1 --2- 172.21.15.122:0/794081594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2168003980 0x7f2168003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f217affd640 1 -- 172.21.15.122:0/794081594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f217c148f40 msgr2=0x7f2168005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:40.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f217a7fc640 1 --2- 172.21.15.122:0/794081594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21680047d0 0x7f2168004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f217affd640 1 --2- 172.21.15.122:0/794081594 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f217c148f40 0x7f2168005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f217affd640 1 -- 172.21.15.122:0/794081594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21680047d0 msgr2=0x7f2168004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:40.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f217affd640 1 --2- 172.21.15.122:0/794081594 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f21680047d0 0x7f2168004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.859+0000 7f217affd640 1 -- 172.21.15.122:0/794081594 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2168003530 con 0x7f2168003980 2024-03-16T20:45:40.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f217affd640 1 --2- 172.21.15.122:0/794081594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2168003980 0x7f2168003dc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2174009fc0 tx=0x7f217401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9dda708068f2256 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:40.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f2179ffb640 1 -- 172.21.15.122:0/794081594 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21740a5a60 con 0x7f2168003980 2024-03-16T20:45:40.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f2179ffb640 1 -- 172.21.15.122:0/794081594 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217400f040 con 0x7f2168003980 2024-03-16T20:45:40.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f2179ffb640 1 -- 172.21.15.122:0/794081594 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21740a5e00 con 0x7f2168003980 2024-03-16T20:45:40.868 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f2181309640 1 -- 172.21.15.122:0/794081594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2168003980 msgr2=0x7f2168003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f2181309640 1 --2- 172.21.15.122:0/794081594 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2168003980 0x7f2168003dc0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2174009fc0 tx=0x7f217401ccc0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f217b7fe640 1 -- 172.21.15.122:0/794081594 reap_dead start 2024-03-16T20:45:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f2181309640 1 -- 172.21.15.122:0/794081594 shutdown_connections 2024-03-16T20:45:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f2181309640 1 -- 172.21.15.122:0/794081594 >> 172.21.15.122:0/794081594 conn(0x7f217c0b7350 msgr2=0x7f217c0b7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f2181309640 1 -- 172.21.15.122:0/794081594 shutdown_connections 2024-03-16T20:45:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f2181309640 1 -- 172.21.15.122:0/794081594 wait complete. 2024-03-16T20:45:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f2181309640 1 Processor -- start 2024-03-16T20:45:40.869 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f2181309640 1 -- start start 2024-03-16T20:45:40.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f2181309640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2168003980 0x7f217c14f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f2181309640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f217c145120 0x7f217c1dfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.871 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f2181309640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f217c1e3420 0x7f217c1e38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f2181309640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c145be0 con 0x7f217c145120 2024-03-16T20:45:40.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f2181309640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c144c90 con 0x7f217c1e3420 2024-03-16T20:45:40.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f2181309640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c1410c0 con 0x7f2168003980 2024-03-16T20:45:40.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f217a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f217c145120 0x7f217c1dfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f217a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f217c145120 0x7f217c1dfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42204/0 (socket says 172.21.15.122:42204) 2024-03-16T20:45:40.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f217a7fc640 1 -- 172.21.15.122:0/2897891171 learned_addr learned my addr 172.21.15.122:0/2897891171 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:40.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f217b7fe640 1 --2- 172.21.15.122:0/2897891171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f217c1e3420 0x7f217c1e38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f217a7fc640 1 -- 172.21.15.122:0/2897891171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2168003980 msgr2=0x7f217c14f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:40.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f217affd640 1 --2- 172.21.15.122:0/2897891171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2168003980 0x7f217c14f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f217a7fc640 1 --2- 172.21.15.122:0/2897891171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2168003980 0x7f217c14f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f217a7fc640 1 -- 172.21.15.122:0/2897891171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f217c1e3420 msgr2=0x7f217c1e38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:40.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f217a7fc640 1 --2- 172.21.15.122:0/2897891171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f217c1e3420 0x7f217c1e38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:40.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f217a7fc640 1 -- 172.21.15.122:0/2897891171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c141330 con 0x7f217c145120 2024-03-16T20:45:40.872 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f217b7fe640 1 --2- 172.21.15.122:0/2897891171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f217c1e3420 0x7f217c1e38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:40.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f217a7fc640 1 --2- 172.21.15.122:0/2897891171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f217c145120 0x7f217c1dfeb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f21640018f0 tx=0x7f216400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:40.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.863+0000 7f21737fe640 1 -- 172.21.15.122:0/2897891171 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21640074e0 con 0x7f217c145120 2024-03-16T20:45:40.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.867+0000 7f21737fe640 1 -- 172.21.15.122:0/2897891171 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2164007640 con 0x7f217c145120 2024-03-16T20:45:40.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.867+0000 7f2181309640 1 -- 172.21.15.122:0/2897891171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f217c145900 con 0x7f217c145120 2024-03-16T20:45:40.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.867+0000 7f21737fe640 1 -- 172.21.15.122:0/2897891171 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164004a10 con 0x7f217c145120 2024-03-16T20:45:40.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.867+0000 7f2181309640 1 -- 172.21.15.122:0/2897891171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f217c1e03f0 con 0x7f217c145120 2024-03-16T20:45:40.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.867+0000 7f2181309640 1 -- 172.21.15.122:0/2897891171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217c144e00 con 0x7f217c145120 2024-03-16T20:45:40.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.867+0000 7f21737fe640 1 -- 172.21.15.122:0/2897891171 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f2164004b70 con 0x7f217c145120 2024-03-16T20:45:40.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.867+0000 7f21737fe640 1 --2- 172.21.15.122:0/2897891171 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f215c066e70 0x7f215c069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:40.873 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.867+0000 7f21737fe640 1 -- 172.21.15.122:0/2897891171 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2164084330 con 0x7f217c145120 2024-03-16T20:45:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.867+0000 7f217affd640 1 --2- 172.21.15.122:0/2897891171 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f215c066e70 0x7f215c069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:40.874 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.867+0000 7f217affd640 1 --2- 172.21.15.122:0/2897891171 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f215c066e70 0x7f215c069330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f21740af000 tx=0x7f21740a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:40.875 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:40.871+0000 7f21737fe640 1 -- 172.21.15.122:0/2897891171 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2164051500 con 0x7f217c145120 2024-03-16T20:45:41.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.087+0000 7f2181309640 1 -- 172.21.15.122:0/2897891171 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f217c074bb0 con 0x7f217c145120 2024-03-16T20:45:41.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.091+0000 7f21737fe640 1 -- 172.21.15.122:0/2897891171 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2164054bc0 con 0x7f217c145120 2024-03-16T20:45:41.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.095+0000 7f2181309640 1 -- 172.21.15.122:0/2897891171 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f215c066e70 msgr2=0x7f215c069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:41.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.095+0000 7f2181309640 1 --2- 172.21.15.122:0/2897891171 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f215c066e70 0x7f215c069330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f21740af000 tx=0x7f21740a7040 comp rx=0 tx=0).stop 2024-03-16T20:45:41.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.095+0000 7f2181309640 1 -- 172.21.15.122:0/2897891171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f217c145120 msgr2=0x7f217c1dfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:41.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.095+0000 7f2181309640 1 --2- 172.21.15.122:0/2897891171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f217c145120 0x7f217c1dfeb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f21640018f0 tx=0x7f216400dc90 comp rx=0 tx=0).stop 2024-03-16T20:45:41.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.095+0000 7f2181309640 1 -- 172.21.15.122:0/2897891171 shutdown_connections 2024-03-16T20:45:41.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.095+0000 7f2181309640 1 --2- 172.21.15.122:0/2897891171 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f215c066e70 0x7f215c069330 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.095+0000 7f2181309640 1 --2- 172.21.15.122:0/2897891171 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f217c1e3420 0x7f217c1e38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.095+0000 7f2181309640 1 --2- 172.21.15.122:0/2897891171 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f217c145120 0x7f217c1dfeb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.095+0000 7f2181309640 1 --2- 172.21.15.122:0/2897891171 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2168003980 0x7f217c14f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.095+0000 7f2181309640 1 -- 172.21.15.122:0/2897891171 >> 172.21.15.122:0/2897891171 conn(0x7f217c0b7350 msgr2=0x7f217c14ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:41.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.095+0000 7f2181309640 1 -- 172.21.15.122:0/2897891171 shutdown_connections 2024-03-16T20:45:41.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.095+0000 7f2181309640 1 -- 172.21.15.122:0/2897891171 wait complete. 2024-03-16T20:45:41.115 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-03-16T20:45:41.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff35a647640 1 Processor -- start 2024-03-16T20:45:41.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff35a647640 1 -- start start 2024-03-16T20:45:41.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff35a647640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff354148f00 0x7ff35413ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff35a647640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff35413f320 0x7ff35414f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff35a647640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3540b3480 con 0x7ff3540b30a0 2024-03-16T20:45:41.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff35a647640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3540b83a0 con 0x7ff354148f00 2024-03-16T20:45:41.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff35a647640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3540b8680 con 0x7ff35413f320 2024-03-16T20:45:41.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff358be4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff35413f320 0x7ff35414f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff34bfff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff354148f00 0x7ff35413ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff353fff640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7ff3540b30a0 0x7ff3541487f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.122:34824/0 (socket says 172.21.15.122:34824) 2024-03-16T20:45:41.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff358be4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff35413f320 0x7ff35414f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:55838/0 (socket says 172.21.15.122:55838) 2024-03-16T20:45:41.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff353fff640 1 -- 172.21.15.122:0/2544658329 learned_addr learned my addr 172.21.15.122:0/2544658329 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:41.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff34bfff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff354148f00 0x7ff35413ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.122:46444/0 (socket says 172.21.15.122:46444) 2024-03-16T20:45:41.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff34bfff640 1 -- 172.21.15.122:0/2544658329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff35413f320 msgr2=0x7ff35414f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:41.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff34bfff640 1 --2- 172.21.15.122:0/2544658329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff35413f320 0x7ff35414f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff34bfff640 1 -- 172.21.15.122:0/2544658329 >> v1:172.21.15.195:6789/0 conn(0x7ff3540b30a0 legacy=0x7ff3541487f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:41.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff34bfff640 1 -- 172.21.15.122:0/2544658329 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3540b8960 con 0x7ff354148f00 2024-03-16T20:45:41.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.231+0000 7ff358be4640 1 --2- 172.21.15.122:0/2544658329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff35413f320 0x7ff35414f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:45:41.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff34bfff640 1 --2- 172.21.15.122:0/2544658329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff354148f00 0x7ff35413ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0097c0 tx=0x7ff33c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8540c139279a59b4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:41.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff3537fe640 1 -- 172.21.15.122:0/2544658329 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0ab600 con 0x7ff354148f00 2024-03-16T20:45:41.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff3537fe640 1 -- 172.21.15.122:0/2544658329 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33c0a7050 con 0x7ff354148f00 2024-03-16T20:45:41.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff3537fe640 1 -- 172.21.15.122:0/2544658329 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33c0ab980 con 0x7ff354148f00 2024-03-16T20:45:41.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff35a647640 1 -- 172.21.15.122:0/2544658329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff354148f00 msgr2=0x7ff35413ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:41.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff35a647640 1 --2- 172.21.15.122:0/2544658329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff354148f00 0x7ff35413ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0097c0 tx=0x7ff33c01c410 comp rx=0 tx=0).stop 2024-03-16T20:45:41.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff35a647640 1 -- 172.21.15.122:0/2544658329 shutdown_connections 2024-03-16T20:45:41.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff35a647640 1 --2- 172.21.15.122:0/2544658329 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff35413f320 0x7ff35414f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff35a647640 1 --2- 172.21.15.122:0/2544658329 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff354148f00 0x7ff35413ede0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff35a647640 1 -- 172.21.15.122:0/2544658329 >> 172.21.15.122:0/2544658329 conn(0x7ff354070d30 msgr2=0x7ff354071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:41.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff35a647640 1 -- 172.21.15.122:0/2544658329 shutdown_connections 2024-03-16T20:45:41.240 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff35a647640 1 -- 172.21.15.122:0/2544658329 wait complete. 2024-03-16T20:45:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff35a647640 1 Processor -- start 2024-03-16T20:45:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff35a647640 1 -- start start 2024-03-16T20:45:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff35a647640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3540b30a0 0x7ff354136ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff35a647640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff35413f320 0x7ff3540b51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff35a647640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff354148f00 0x7ff3540b5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff35a647640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff354136070 con 0x7ff3540b30a0 2024-03-16T20:45:41.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff35a647640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff354136350 con 0x7ff354148f00 2024-03-16T20:45:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff35a647640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3540b5c60 con 0x7ff35413f320 2024-03-16T20:45:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff353fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3540b30a0 0x7ff354136ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff353fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3540b30a0 0x7ff354136ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42210/0 (socket says 172.21.15.122:42210) 2024-03-16T20:45:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff353fff640 1 -- 172.21.15.122:0/9162557 learned_addr learned my addr 172.21.15.122:0/9162557 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff358be4640 1 --2- 172.21.15.122:0/9162557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff354148f00 0x7ff3540b5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff34bfff640 1 --2- 172.21.15.122:0/9162557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff35413f320 0x7ff3540b51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff353fff640 1 -- 172.21.15.122:0/9162557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff35413f320 msgr2=0x7ff3540b51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff353fff640 1 --2- 172.21.15.122:0/9162557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff35413f320 0x7ff3540b51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff353fff640 1 -- 172.21.15.122:0/9162557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff354148f00 msgr2=0x7ff3540b5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff353fff640 1 --2- 172.21.15.122:0/9162557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff354148f00 0x7ff3540b5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff353fff640 1 -- 172.21.15.122:0/9162557 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34c009a70 con 0x7ff3540b30a0 2024-03-16T20:45:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff34bfff640 1 --2- 172.21.15.122:0/9162557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff35413f320 0x7ff3540b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:41.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff353fff640 1 --2- 172.21.15.122:0/9162557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3540b30a0 0x7ff354136ab0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff34400c3a0 tx=0x7ff34400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:41.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff3517fa640 1 -- 172.21.15.122:0/9162557 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff344005cf0 con 0x7ff3540b30a0 2024-03-16T20:45:41.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff3517fa640 1 -- 172.21.15.122:0/9162557 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff344005e50 con 0x7ff3540b30a0 2024-03-16T20:45:41.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff35a647640 1 -- 172.21.15.122:0/9162557 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33c009420 con 0x7ff3540b30a0 2024-03-16T20:45:41.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.235+0000 7ff3517fa640 1 -- 172.21.15.122:0/9162557 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3440072f0 con 0x7ff3540b30a0 2024-03-16T20:45:41.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.239+0000 7ff35a647640 1 -- 172.21.15.122:0/9162557 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35413e550 con 0x7ff3540b30a0 2024-03-16T20:45:41.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.239+0000 7ff35a647640 1 -- 172.21.15.122:0/9162557 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3540ba6b0 con 0x7ff3540b30a0 2024-03-16T20:45:41.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.239+0000 7ff3517fa640 1 -- 172.21.15.122:0/9162557 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7ff344007450 con 0x7ff3540b30a0 2024-03-16T20:45:41.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.239+0000 7ff3517fa640 1 --2- 172.21.15.122:0/9162557 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff334066e70 0x7ff334069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.239+0000 7ff3517fa640 1 -- 172.21.15.122:0/9162557 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff344083c10 con 0x7ff3540b30a0 2024-03-16T20:45:41.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.239+0000 7ff34bfff640 1 --2- 172.21.15.122:0/9162557 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff334066e70 0x7ff334069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.239+0000 7ff34bfff640 1 --2- 172.21.15.122:0/9162557 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff334066e70 0x7ff334069330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0ad000 tx=0x7ff33c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:41.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.243+0000 7ff3517fa640 1 -- 172.21.15.122:0/9162557 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff344050de0 con 0x7ff3540b30a0 2024-03-16T20:45:41.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.443+0000 7ff35a647640 1 -- 172.21.15.122:0/9162557 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7ff354074bb0 con 0x7ff3540b30a0 2024-03-16T20:45:41.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.447+0000 7ff3517fa640 1 -- 172.21.15.122:0/9162557 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff3440544a0 con 0x7ff3540b30a0 2024-03-16T20:45:41.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.447+0000 7ff35a647640 1 -- 172.21.15.122:0/9162557 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff334066e70 msgr2=0x7ff334069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:41.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.447+0000 7ff35a647640 1 --2- 172.21.15.122:0/9162557 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff334066e70 0x7ff334069330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff33c0ad000 tx=0x7ff33c008040 comp rx=0 tx=0).stop 2024-03-16T20:45:41.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.447+0000 7ff35a647640 1 -- 172.21.15.122:0/9162557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3540b30a0 msgr2=0x7ff354136ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:41.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.447+0000 7ff35a647640 1 --2- 172.21.15.122:0/9162557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3540b30a0 0x7ff354136ab0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff34400c3a0 tx=0x7ff34400c860 comp rx=0 tx=0).stop 2024-03-16T20:45:41.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.451+0000 7ff35a647640 1 -- 172.21.15.122:0/9162557 shutdown_connections 2024-03-16T20:45:41.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.451+0000 7ff35a647640 1 --2- 172.21.15.122:0/9162557 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff334066e70 0x7ff334069330 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.451+0000 7ff35a647640 1 --2- 172.21.15.122:0/9162557 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff354148f00 0x7ff3540b5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.451+0000 7ff35a647640 1 --2- 172.21.15.122:0/9162557 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff35413f320 0x7ff3540b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.451+0000 7ff35a647640 1 --2- 172.21.15.122:0/9162557 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff3540b30a0 0x7ff354136ab0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.451+0000 7ff35a647640 1 -- 172.21.15.122:0/9162557 >> 172.21.15.122:0/9162557 conn(0x7ff354070d30 msgr2=0x7ff354071cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:41.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.451+0000 7ff35a647640 1 -- 172.21.15.122:0/9162557 shutdown_connections 2024-03-16T20:45:41.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.451+0000 7ff35a647640 1 -- 172.21.15.122:0/9162557 wait complete. 2024-03-16T20:45:41.469 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:41.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.583+0000 7fd04d341640 1 Processor -- start 2024-03-16T20:45:41.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.583+0000 7fd04d341640 1 -- start start 2024-03-16T20:45:41.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.583+0000 7fd04d341640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd04806b1f0 0x7fd048070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.583+0000 7fd04d341640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd048157f30 0x7fd04815e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.583+0000 7fd04d341640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0480b44a0 con 0x7fd0480716f0 2024-03-16T20:45:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.583+0000 7fd04d341640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd048158380 con 0x7fd048157f30 2024-03-16T20:45:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.583+0000 7fd04d341640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04806b5d0 con 0x7fd04806b1f0 2024-03-16T20:45:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd0467fc640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fd0480716f0 0x7fd048157820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39484/0 (socket says 172.21.15.122:39484) 2024-03-16T20:45:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd0467fc640 1 -- 172.21.15.122:0/1767677312 learned_addr learned my addr 172.21.15.122:0/1767677312 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd0477fe640 1 --2- 172.21.15.122:0/1767677312 >> v2:172.21.15.122:3300/0 conn(0x7fd048157f30 0x7fd04815e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd046ffd640 1 --2- 172.21.15.122:0/1767677312 >> v2:172.21.15.195:3300/0 conn(0x7fd04806b1f0 0x7fd048070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd0477fe640 1 -- 172.21.15.122:0/1767677312 >> v2:172.21.15.195:3300/0 conn(0x7fd04806b1f0 msgr2=0x7fd048070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd0477fe640 1 --2- 172.21.15.122:0/1767677312 >> v2:172.21.15.195:3300/0 conn(0x7fd04806b1f0 0x7fd048070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd0477fe640 1 -- 172.21.15.122:0/1767677312 >> v1:172.21.15.122:6789/0 conn(0x7fd0480716f0 legacy=0x7fd048157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:41.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd0477fe640 1 -- 172.21.15.122:0/1767677312 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0480afb90 con 0x7fd048157f30 2024-03-16T20:45:41.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd0477fe640 1 --2- 172.21.15.122:0/1767677312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd048157f30 0x7fd04815e700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd04806bd60 tx=0x7fd04001d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f7d01cc723ca4e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:41.592 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd045ffb640 1 -- 172.21.15.122:0/1767677312 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0400ac620 con 0x7fd048157f30 2024-03-16T20:45:41.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd045ffb640 1 -- 172.21.15.122:0/1767677312 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0400ac780 con 0x7fd048157f30 2024-03-16T20:45:41.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd045ffb640 1 -- 172.21.15.122:0/1767677312 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0400acb10 con 0x7fd048157f30 2024-03-16T20:45:41.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd04d341640 1 -- 172.21.15.122:0/1767677312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd048157f30 msgr2=0x7fd04815e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:41.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd04d341640 1 --2- 172.21.15.122:0/1767677312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd048157f30 0x7fd04815e700 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd04806bd60 tx=0x7fd04001d320 comp rx=0 tx=0).stop 2024-03-16T20:45:41.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd04d341640 1 -- 172.21.15.122:0/1767677312 shutdown_connections 2024-03-16T20:45:41.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd04d341640 1 --2- 172.21.15.122:0/1767677312 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd048157f30 0x7fd04815e700 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd04d341640 1 --2- 172.21.15.122:0/1767677312 >> v2:172.21.15.195:3300/0 conn(0x7fd04806b1f0 0x7fd048070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd04d341640 1 -- 172.21.15.122:0/1767677312 >> 172.21.15.122:0/1767677312 conn(0x7fd04806d6b0 msgr2=0x7fd0480b8690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:41.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd04d341640 1 -- 172.21.15.122:0/1767677312 shutdown_connections 2024-03-16T20:45:41.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.587+0000 7fd04d341640 1 -- 172.21.15.122:0/1767677312 wait complete. 2024-03-16T20:45:41.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd04d341640 1 Processor -- start 2024-03-16T20:45:41.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd04d341640 1 -- start start 2024-03-16T20:45:41.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd04d341640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd04806b1f0 0x7fd048151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd04d341640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0480716f0 0x7fd048155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd04d341640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd048157f30 0x7fd048155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd04d341640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd048155cd0 con 0x7fd048157f30 2024-03-16T20:45:41.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd04d341640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd048154e20 con 0x7fd0480716f0 2024-03-16T20:45:41.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd04d341640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd048152310 con 0x7fd04806b1f0 2024-03-16T20:45:41.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd0477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd048157f30 0x7fd048155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd0477fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd048157f30 0x7fd048155790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42230/0 (socket says 172.21.15.122:42230) 2024-03-16T20:45:41.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd0477fe640 1 -- 172.21.15.122:0/1248489065 learned_addr learned my addr 172.21.15.122:0/1248489065 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:41.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd0467fc640 1 --2- 172.21.15.122:0/1248489065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0480716f0 0x7fd048155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd046ffd640 1 --2- 172.21.15.122:0/1248489065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd04806b1f0 0x7fd048151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd0477fe640 1 -- 172.21.15.122:0/1248489065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd04806b1f0 msgr2=0x7fd048151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:41.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd0477fe640 1 --2- 172.21.15.122:0/1248489065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd04806b1f0 0x7fd048151dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd0477fe640 1 -- 172.21.15.122:0/1248489065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0480716f0 msgr2=0x7fd048155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:41.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd0477fe640 1 --2- 172.21.15.122:0/1248489065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0480716f0 0x7fd048155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd0477fe640 1 -- 172.21.15.122:0/1248489065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04000c300 con 0x7fd048157f30 2024-03-16T20:45:41.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd0467fc640 1 --2- 172.21.15.122:0/1248489065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0480716f0 0x7fd048155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:41.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd0477fe640 1 --2- 172.21.15.122:0/1248489065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd048157f30 0x7fd048155790 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd040019f30 tx=0x7fd04001a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:41.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd02f7fe640 1 -- 172.21.15.122:0/1248489065 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0400ace20 con 0x7fd048157f30 2024-03-16T20:45:41.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd02f7fe640 1 -- 172.21.15.122:0/1248489065 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd040009450 con 0x7fd048157f30 2024-03-16T20:45:41.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd04d341640 1 -- 172.21.15.122:0/1248489065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0481525b0 con 0x7fd048157f30 2024-03-16T20:45:41.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd02f7fe640 1 -- 172.21.15.122:0/1248489065 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd040009800 con 0x7fd048157f30 2024-03-16T20:45:41.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd04d341640 1 -- 172.21.15.122:0/1248489065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd048153020 con 0x7fd048157f30 2024-03-16T20:45:41.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd04d341640 1 -- 172.21.15.122:0/1248489065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd048075280 con 0x7fd048157f30 2024-03-16T20:45:41.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd02f7fe640 1 -- 172.21.15.122:0/1248489065 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fd040009960 con 0x7fd048157f30 2024-03-16T20:45:41.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd02f7fe640 1 --2- 172.21.15.122:0/1248489065 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd034066e40 0x7fd034069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.591+0000 7fd02f7fe640 1 -- 172.21.15.122:0/1248489065 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd0400bc030 con 0x7fd048157f30 2024-03-16T20:45:41.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.595+0000 7fd046ffd640 1 --2- 172.21.15.122:0/1248489065 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd034066e40 0x7fd034069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.595+0000 7fd046ffd640 1 --2- 172.21.15.122:0/1248489065 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd034066e40 0x7fd034069300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009790 tx=0x7fd03c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:41.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.599+0000 7fd02f7fe640 1 -- 172.21.15.122:0/1248489065 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd0400ecbc0 con 0x7fd048157f30 2024-03-16T20:45:41.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.743+0000 7fd04d341640 1 -- 172.21.15.122:0/1248489065 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd04808a400 con 0x7fd048157f30 2024-03-16T20:45:41.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.743+0000 7fd02f7fe640 1 -- 172.21.15.122:0/1248489065 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+12927 (secure 0 0 0) 0x7fd0400c6020 con 0x7fd048157f30 2024-03-16T20:45:41.749 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:41.749 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":26,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:38.891607+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:41.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.747+0000 7fd04d341640 1 -- 172.21.15.122:0/1248489065 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd034066e40 msgr2=0x7fd034069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:41.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.747+0000 7fd04d341640 1 --2- 172.21.15.122:0/1248489065 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd034066e40 0x7fd034069300 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd03c009790 tx=0x7fd03c008040 comp rx=0 tx=0).stop 2024-03-16T20:45:41.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.747+0000 7fd04d341640 1 -- 172.21.15.122:0/1248489065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd048157f30 msgr2=0x7fd048155790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:41.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.747+0000 7fd04d341640 1 --2- 172.21.15.122:0/1248489065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd048157f30 0x7fd048155790 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd040019f30 tx=0x7fd04001a0c0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.747+0000 7fd04d341640 1 -- 172.21.15.122:0/1248489065 shutdown_connections 2024-03-16T20:45:41.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.747+0000 7fd04d341640 1 --2- 172.21.15.122:0/1248489065 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd034066e40 0x7fd034069300 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.747+0000 7fd04d341640 1 --2- 172.21.15.122:0/1248489065 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd048157f30 0x7fd048155790 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.747+0000 7fd04d341640 1 --2- 172.21.15.122:0/1248489065 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd0480716f0 0x7fd048155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.747+0000 7fd04d341640 1 --2- 172.21.15.122:0/1248489065 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd04806b1f0 0x7fd048151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.747+0000 7fd04d341640 1 -- 172.21.15.122:0/1248489065 >> 172.21.15.122:0/1248489065 conn(0x7fd04806d6b0 msgr2=0x7fd0480b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:41.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.747+0000 7fd04d341640 1 -- 172.21.15.122:0/1248489065 shutdown_connections 2024-03-16T20:45:41.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.747+0000 7fd04d341640 1 -- 172.21.15.122:0/1248489065 wait complete. 2024-03-16T20:45:41.766 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T20:44:52.653523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T20:45:41.767 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T20:45:41.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.883+0000 7f6f6d40e640 1 Processor -- start 2024-03-16T20:45:41.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.887+0000 7f6f6d40e640 1 -- start start 2024-03-16T20:45:41.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.887+0000 7f6f6d40e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6f680bb1d0 0x7f6f6806cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.887+0000 7f6f6d40e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6f6806d180 0x7f6f68151920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.887+0000 7f6f6d40e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f68147ff0 con 0x7f6f68141580 2024-03-16T20:45:41.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.887+0000 7f6f6d40e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f681479a0 con 0x7f6f680bb1d0 2024-03-16T20:45:41.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.887+0000 7f6f6d40e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f68147c80 con 0x7f6f6806d180 2024-03-16T20:45:41.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.887+0000 7f6f5e7fc640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6f680bb1d0 0x7f6f6806cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.887+0000 7f6f5e7fc640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6f680bb1d0 0x7f6f6806cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42234/0 (socket says 172.21.15.122:42234) 2024-03-16T20:45:41.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.887+0000 7f6f5e7fc640 1 -- 172.21.15.122:0/1627874270 learned_addr learned my addr 172.21.15.122:0/1627874270 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:41.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.887+0000 7f6f677fe640 1 --2- 172.21.15.122:0/1627874270 >> v2:172.21.15.195:3300/0 conn(0x7f6f6806d180 0x7f6f68151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.887+0000 7f6f5e7fc640 1 -- 172.21.15.122:0/1627874270 >> v2:172.21.15.195:3300/0 conn(0x7f6f6806d180 msgr2=0x7f6f68151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:41.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.887+0000 7f6f5e7fc640 1 --2- 172.21.15.122:0/1627874270 >> v2:172.21.15.195:3300/0 conn(0x7f6f6806d180 0x7f6f68151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.887+0000 7f6f5e7fc640 1 -- 172.21.15.122:0/1627874270 >> v1:172.21.15.195:6789/0 conn(0x7f6f68141580 legacy=0x7f6f680baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:41.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.887+0000 7f6f5e7fc640 1 -- 172.21.15.122:0/1627874270 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f681475b0 con 0x7f6f680bb1d0 2024-03-16T20:45:41.892 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.887+0000 7f6f677fe640 1 --2- 172.21.15.122:0/1627874270 >> v2:172.21.15.195:3300/0 conn(0x7f6f6806d180 0x7f6f68151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:41.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f5e7fc640 1 --2- 172.21.15.122:0/1627874270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f680bb1d0 0x7f6f6806cc40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6f680bc7b0 tx=0x7f6f5001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cc47af6bbc92c0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:41.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f667fc640 1 -- 172.21.15.122:0/1627874270 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f500ab600 con 0x7f6f680bb1d0 2024-03-16T20:45:41.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f667fc640 1 -- 172.21.15.122:0/1627874270 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f500a7050 con 0x7f6f680bb1d0 2024-03-16T20:45:41.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f667fc640 1 -- 172.21.15.122:0/1627874270 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f500ab980 con 0x7f6f680bb1d0 2024-03-16T20:45:41.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 -- 172.21.15.122:0/1627874270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f680bb1d0 msgr2=0x7f6f6806cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:41.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 --2- 172.21.15.122:0/1627874270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f680bb1d0 0x7f6f6806cc40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6f680bc7b0 tx=0x7f6f5001c4e0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 -- 172.21.15.122:0/1627874270 shutdown_connections 2024-03-16T20:45:41.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 --2- 172.21.15.122:0/1627874270 >> v2:172.21.15.195:3300/0 conn(0x7f6f6806d180 0x7f6f68151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 --2- 172.21.15.122:0/1627874270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f680bb1d0 0x7f6f6806cc40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 -- 172.21.15.122:0/1627874270 >> 172.21.15.122:0/1627874270 conn(0x7f6f680bca20 msgr2=0x7f6f680bce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:41.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 -- 172.21.15.122:0/1627874270 shutdown_connections 2024-03-16T20:45:41.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 -- 172.21.15.122:0/1627874270 wait complete. 2024-03-16T20:45:41.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 Processor -- start 2024-03-16T20:45:41.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 -- start start 2024-03-16T20:45:41.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f6806d180 0x7f6f680b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f680bb1d0 0x7f6f68144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f68141580 0x7f6f6813a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680b6050 con 0x7f6f6806d180 2024-03-16T20:45:41.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680b5af0 con 0x7f6f680bb1d0 2024-03-16T20:45:41.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f68141b70 con 0x7f6f68141580 2024-03-16T20:45:41.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f66ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f6806d180 0x7f6f680b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f66ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f6806d180 0x7f6f680b55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42248/0 (socket says 172.21.15.122:42248) 2024-03-16T20:45:41.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f66ffd640 1 -- 172.21.15.122:0/1498685905 learned_addr learned my addr 172.21.15.122:0/1498685905 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:41.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f5e7fc640 1 --2- 172.21.15.122:0/1498685905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f680bb1d0 0x7f6f68144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f677fe640 1 --2- 172.21.15.122:0/1498685905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f68141580 0x7f6f6813a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f66ffd640 1 -- 172.21.15.122:0/1498685905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f68141580 msgr2=0x7f6f6813a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:41.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f66ffd640 1 --2- 172.21.15.122:0/1498685905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f68141580 0x7f6f6813a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f66ffd640 1 -- 172.21.15.122:0/1498685905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f680bb1d0 msgr2=0x7f6f68144ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:41.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f66ffd640 1 --2- 172.21.15.122:0/1498685905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f680bb1d0 0x7f6f68144ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:41.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f66ffd640 1 -- 172.21.15.122:0/1498685905 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f500094d0 con 0x7f6f6806d180 2024-03-16T20:45:41.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f677fe640 1 --2- 172.21.15.122:0/1498685905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f68141580 0x7f6f6813a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:41.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f66ffd640 1 --2- 172.21.15.122:0/1498685905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f6806d180 0x7f6f680b55b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800b410 tx=0x7f6f5800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:41.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f5ffff640 1 -- 172.21.15.122:0/1498685905 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f58004020 con 0x7f6f6806d180 2024-03-16T20:45:41.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f5ffff640 1 -- 172.21.15.122:0/1498685905 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f58004180 con 0x7f6f6806d180 2024-03-16T20:45:41.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 -- 172.21.15.122:0/1498685905 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f680b5d50 con 0x7f6f6806d180 2024-03-16T20:45:41.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f5ffff640 1 -- 172.21.15.122:0/1498685905 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f58011630 con 0x7f6f6806d180 2024-03-16T20:45:41.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.891+0000 7f6f6d40e640 1 -- 172.21.15.122:0/1498685905 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f681e3190 con 0x7f6f6806d180 2024-03-16T20:45:41.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.895+0000 7f6f6d40e640 1 -- 172.21.15.122:0/1498685905 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f681e36c0 con 0x7f6f6806d180 2024-03-16T20:45:41.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.895+0000 7f6f5ffff640 1 -- 172.21.15.122:0/1498685905 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f6f58011790 con 0x7f6f6806d180 2024-03-16T20:45:41.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.895+0000 7f6f5ffff640 1 --2- 172.21.15.122:0/1498685905 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6f48066e70 0x7f6f48069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:41.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.895+0000 7f6f5ffff640 1 -- 172.21.15.122:0/1498685905 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6f58084ad0 con 0x7f6f6806d180 2024-03-16T20:45:41.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.895+0000 7f6f5e7fc640 1 --2- 172.21.15.122:0/1498685905 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6f48066e70 0x7f6f48069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:41.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.895+0000 7f6f5e7fc640 1 --2- 172.21.15.122:0/1498685905 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6f48066e70 0x7f6f48069330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6f500ad000 tx=0x7f6f50008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:41.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:41.899+0000 7f6f5ffff640 1 -- 172.21.15.122:0/1498685905 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6f58051d80 con 0x7f6f6806d180 2024-03-16T20:45:42.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.055+0000 7f6f6d40e640 1 -- 172.21.15.122:0/1498685905 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f68074bb0 con 0x7f6f6806d180 2024-03-16T20:45:42.062 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.059+0000 7f6f5ffff640 1 -- 172.21.15.122:0/1498685905 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f58055440 con 0x7f6f6806d180 2024-03-16T20:45:42.062 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-03-16T20:45:42.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.059+0000 7f6f6d40e640 1 -- 172.21.15.122:0/1498685905 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6f48066e70 msgr2=0x7f6f48069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:42.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.059+0000 7f6f6d40e640 1 --2- 172.21.15.122:0/1498685905 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6f48066e70 0x7f6f48069330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6f500ad000 tx=0x7f6f50008040 comp rx=0 tx=0).stop 2024-03-16T20:45:42.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.059+0000 7f6f6d40e640 1 -- 172.21.15.122:0/1498685905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f6806d180 msgr2=0x7f6f680b55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:42.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.059+0000 7f6f6d40e640 1 --2- 172.21.15.122:0/1498685905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f6806d180 0x7f6f680b55b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800b410 tx=0x7f6f5800b8d0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.059+0000 7f6f6d40e640 1 -- 172.21.15.122:0/1498685905 shutdown_connections 2024-03-16T20:45:42.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.059+0000 7f6f6d40e640 1 --2- 172.21.15.122:0/1498685905 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6f48066e70 0x7f6f48069330 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.059+0000 7f6f6d40e640 1 --2- 172.21.15.122:0/1498685905 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6f68141580 0x7f6f6813a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.059+0000 7f6f6d40e640 1 --2- 172.21.15.122:0/1498685905 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6f680bb1d0 0x7f6f68144ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.059+0000 7f6f6d40e640 1 --2- 172.21.15.122:0/1498685905 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6f6806d180 0x7f6f680b55b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.059+0000 7f6f6d40e640 1 -- 172.21.15.122:0/1498685905 >> 172.21.15.122:0/1498685905 conn(0x7f6f680bca20 msgr2=0x7f6f6813fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:42.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.059+0000 7f6f6d40e640 1 -- 172.21.15.122:0/1498685905 shutdown_connections 2024-03-16T20:45:42.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.059+0000 7f6f6d40e640 1 -- 172.21.15.122:0/1498685905 wait complete. 2024-03-16T20:45:42.081 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-16T20:45:42.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.187+0000 7f5e43bdc640 1 Processor -- start 2024-03-16T20:45:42.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.187+0000 7f5e43bdc640 1 -- start start 2024-03-16T20:45:42.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e43bdc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5e3c13ed60 0x7f5e3c13f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:42.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e43bdc640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3c0b02a0 con 0x7f5e3c148430 2024-03-16T20:45:42.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e43bdc640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3c0b1050 con 0x7f5e3c072120 2024-03-16T20:45:42.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e43bdc640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c072500 con 0x7f5e3c13ed60 2024-03-16T20:45:42.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e42179640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f5e3c148430 0x7f5e3c14f780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39496/0 (socket says 172.21.15.122:39496) 2024-03-16T20:45:42.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e42179640 1 -- 172.21.15.122:0/653434500 learned_addr learned my addr 172.21.15.122:0/653434500 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:42.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e40976640 1 -- 172.21.15.122:0/653434500 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2565170505 0 0) 0x7f5e3c0b02a0 con 0x7f5e3c148430 2024-03-16T20:45:42.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e41177640 1 --2- 172.21.15.122:0/653434500 >> v2:172.21.15.195:3300/0 conn(0x7f5e3c13ed60 0x7f5e3c13f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:42.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e40976640 1 -- 172.21.15.122:0/653434500 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e24003530 con 0x7f5e3c148430 2024-03-16T20:45:42.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e40976640 1 -- 172.21.15.122:0/653434500 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1394716960 0 0) 0x7f5e24003530 con 0x7f5e3c148430 2024-03-16T20:45:42.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e40976640 1 -- 172.21.15.122:0/653434500 >> v2:172.21.15.195:3300/0 conn(0x7f5e3c13ed60 msgr2=0x7f5e3c13f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:42.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e40976640 1 --2- 172.21.15.122:0/653434500 >> v2:172.21.15.195:3300/0 conn(0x7f5e3c13ed60 0x7f5e3c13f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e40976640 1 -- 172.21.15.122:0/653434500 >> v1:172.21.15.195:6789/0 conn(0x7f5e3c072120 legacy=0x7f5e3c0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:42.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e40976640 1 -- 172.21.15.122:0/653434500 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c0b38a0 con 0x7f5e3c148430 2024-03-16T20:45:42.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e40976640 1 -- 172.21.15.122:0/653434500 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f5e2c001680 con 0x7f5e3c148430 2024-03-16T20:45:42.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e40976640 1 -- 172.21.15.122:0/653434500 >> v1:172.21.15.122:6789/0 conn(0x7f5e3c148430 legacy=0x7f5e3c14f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:42.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e40976640 1 --2- 172.21.15.122:0/653434500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e24003980 0x7f5e24003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:42.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e40976640 1 --2- 172.21.15.122:0/653434500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e240047d0 0x7f5e24004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:42.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e40976640 1 --2- 172.21.15.122:0/653434500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e24005190 0x7f5e2400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:42.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e40976640 1 -- 172.21.15.122:0/653434500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c072500 con 0x7f5e24003980 2024-03-16T20:45:42.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e40976640 1 -- 172.21.15.122:0/653434500 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2400c130 con 0x7f5e240047d0 2024-03-16T20:45:42.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e40976640 1 -- 172.21.15.122:0/653434500 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2400c2f0 con 0x7f5e24005190 2024-03-16T20:45:42.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e41978640 1 --2- 172.21.15.122:0/653434500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e24003980 0x7f5e24003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:42.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e41177640 1 --2- 172.21.15.122:0/653434500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e240047d0 0x7f5e24004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:42.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e41978640 1 -- 172.21.15.122:0/653434500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e24005190 msgr2=0x7f5e2400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:42.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e41978640 1 --2- 172.21.15.122:0/653434500 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e24005190 0x7f5e2400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e41978640 1 -- 172.21.15.122:0/653434500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e240047d0 msgr2=0x7f5e24004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:42.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e41978640 1 --2- 172.21.15.122:0/653434500 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e240047d0 0x7f5e24004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e41978640 1 -- 172.21.15.122:0/653434500 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e24003530 con 0x7f5e24003980 2024-03-16T20:45:42.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.191+0000 7f5e42179640 1 -- 172.21.15.122:0/653434500 reap_dead start 2024-03-16T20:45:42.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e41978640 1 --2- 172.21.15.122:0/653434500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e24003980 0x7f5e24003da0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5e34001e70 tx=0x7f5e3401c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ac893ac41b401a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:42.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e40976640 1 -- 172.21.15.122:0/653434500 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34003b80 con 0x7f5e24003980 2024-03-16T20:45:42.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e40976640 1 -- 172.21.15.122:0/653434500 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e34003ce0 con 0x7f5e24003980 2024-03-16T20:45:42.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e40976640 1 -- 172.21.15.122:0/653434500 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34005310 con 0x7f5e24003980 2024-03-16T20:45:42.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e43bdc640 1 -- 172.21.15.122:0/653434500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e24003980 msgr2=0x7f5e24003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:42.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e43bdc640 1 --2- 172.21.15.122:0/653434500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e24003980 0x7f5e24003da0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5e34001e70 tx=0x7f5e3401c6d0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e43bdc640 1 -- 172.21.15.122:0/653434500 shutdown_connections 2024-03-16T20:45:42.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e43bdc640 1 --2- 172.21.15.122:0/653434500 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e24003980 0x7f5e24003da0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e43bdc640 1 -- 172.21.15.122:0/653434500 >> 172.21.15.122:0/653434500 conn(0x7f5e3c0b8860 msgr2=0x7f5e3c0b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:42.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e43bdc640 1 -- 172.21.15.122:0/653434500 shutdown_connections 2024-03-16T20:45:42.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e43bdc640 1 -- 172.21.15.122:0/653434500 wait complete. 2024-03-16T20:45:42.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e43bdc640 1 Processor -- start 2024-03-16T20:45:42.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e43bdc640 1 -- start start 2024-03-16T20:45:42.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e43bdc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e24003980 0x7f5e3c13c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:42.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e43bdc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e3c13d170 0x7f5e3c136a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:42.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e43bdc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c136f60 0x7f5e3c1e10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:42.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e43bdc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0b8450 con 0x7f5e3c136f60 2024-03-16T20:45:42.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e43bdc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c13daf0 con 0x7f5e3c13d170 2024-03-16T20:45:42.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e43bdc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c13e390 con 0x7f5e24003980 2024-03-16T20:45:42.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e42179640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c136f60 0x7f5e3c1e10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:42.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e42179640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c136f60 0x7f5e3c1e10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42274/0 (socket says 172.21.15.122:42274) 2024-03-16T20:45:42.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e42179640 1 -- 172.21.15.122:0/274499595 learned_addr learned my addr 172.21.15.122:0/274499595 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:42.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e41177640 1 --2- 172.21.15.122:0/274499595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e3c13d170 0x7f5e3c136a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:42.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e41978640 1 --2- 172.21.15.122:0/274499595 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e24003980 0x7f5e3c13c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:42.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e42179640 1 -- 172.21.15.122:0/274499595 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e24003980 msgr2=0x7f5e3c13c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:42.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e42179640 1 --2- 172.21.15.122:0/274499595 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e24003980 0x7f5e3c13c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e42179640 1 -- 172.21.15.122:0/274499595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e3c13d170 msgr2=0x7f5e3c136a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:42.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e42179640 1 --2- 172.21.15.122:0/274499595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e3c13d170 0x7f5e3c136a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e42179640 1 -- 172.21.15.122:0/274499595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c13e670 con 0x7f5e3c136f60 2024-03-16T20:45:42.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e41978640 1 --2- 172.21.15.122:0/274499595 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e24003980 0x7f5e3c13c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:42.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e42179640 1 --2- 172.21.15.122:0/274499595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c136f60 0x7f5e3c1e10e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c148810 tx=0x7f5e2c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:42.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e3a7fc640 1 -- 172.21.15.122:0/274499595 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c007500 con 0x7f5e3c136f60 2024-03-16T20:45:42.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e3a7fc640 1 -- 172.21.15.122:0/274499595 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c007680 con 0x7f5e3c136f60 2024-03-16T20:45:42.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e3a7fc640 1 -- 172.21.15.122:0/274499595 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c004a30 con 0x7f5e3c136f60 2024-03-16T20:45:42.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e43bdc640 1 -- 172.21.15.122:0/274499595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c0b7ec0 con 0x7f5e3c136f60 2024-03-16T20:45:42.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.195+0000 7f5e43bdc640 1 -- 172.21.15.122:0/274499595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c13d5f0 con 0x7f5e3c136f60 2024-03-16T20:45:42.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.199+0000 7f5e43bdc640 1 -- 172.21.15.122:0/274499595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c13d8a0 con 0x7f5e3c136f60 2024-03-16T20:45:42.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.199+0000 7f5e3a7fc640 1 -- 172.21.15.122:0/274499595 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f5e2c004b90 con 0x7f5e3c136f60 2024-03-16T20:45:42.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.199+0000 7f5e3a7fc640 1 --2- 172.21.15.122:0/274499595 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5e28066e40 0x7f5e28069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:42.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.199+0000 7f5e3a7fc640 1 -- 172.21.15.122:0/274499595 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5e2c083ee0 con 0x7f5e3c136f60 2024-03-16T20:45:42.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.199+0000 7f5e41978640 1 --2- 172.21.15.122:0/274499595 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5e28066e40 0x7f5e28069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:42.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.199+0000 7f5e41978640 1 --2- 172.21.15.122:0/274499595 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5e28066e40 0x7f5e28069300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5e340ad000 tx=0x7f5e34009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:42.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.203+0000 7f5e3a7fc640 1 -- 172.21.15.122:0/274499595 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f5e2c0510b0 con 0x7f5e3c136f60 2024-03-16T20:45:42.356 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.351+0000 7f5e43bdc640 1 -- 172.21.15.122:0/274499595 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5e3c075280 con 0x7f5e3c136f60 2024-03-16T20:45:42.357 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.351+0000 7f5e3a7fc640 1 -- 172.21.15.122:0/274499595 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v26) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5e2c054770 con 0x7f5e3c136f60 2024-03-16T20:45:42.357 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:42.357 INFO:teuthology.orchestra.run.smithi122.stdout:[ 2024-03-16T20:45:42.357 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-03-16T20:45:42.357 INFO:teuthology.orchestra.run.smithi122.stdout: "poolnum": 1, 2024-03-16T20:45:42.357 INFO:teuthology.orchestra.run.smithi122.stdout: "poolname": ".mgr" 2024-03-16T20:45:42.357 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:42.357 INFO:teuthology.orchestra.run.smithi122.stdout:] 2024-03-16T20:45:42.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.355+0000 7f5e43bdc640 1 -- 172.21.15.122:0/274499595 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5e28066e40 msgr2=0x7f5e28069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:42.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.355+0000 7f5e43bdc640 1 --2- 172.21.15.122:0/274499595 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5e28066e40 0x7f5e28069300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5e340ad000 tx=0x7f5e34009040 comp rx=0 tx=0).stop 2024-03-16T20:45:42.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.355+0000 7f5e43bdc640 1 -- 172.21.15.122:0/274499595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c136f60 msgr2=0x7f5e3c1e10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:42.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.355+0000 7f5e43bdc640 1 --2- 172.21.15.122:0/274499595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c136f60 0x7f5e3c1e10e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c148810 tx=0x7f5e2c00c890 comp rx=0 tx=0).stop 2024-03-16T20:45:42.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.355+0000 7f5e43bdc640 1 -- 172.21.15.122:0/274499595 shutdown_connections 2024-03-16T20:45:42.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.355+0000 7f5e43bdc640 1 --2- 172.21.15.122:0/274499595 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5e28066e40 0x7f5e28069300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.355+0000 7f5e43bdc640 1 --2- 172.21.15.122:0/274499595 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e3c136f60 0x7f5e3c1e10e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.355+0000 7f5e43bdc640 1 --2- 172.21.15.122:0/274499595 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5e3c13d170 0x7f5e3c136a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.355+0000 7f5e43bdc640 1 --2- 172.21.15.122:0/274499595 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f5e24003980 0x7f5e3c13c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.355+0000 7f5e43bdc640 1 -- 172.21.15.122:0/274499595 >> 172.21.15.122:0/274499595 conn(0x7f5e3c0b8860 msgr2=0x7f5e3c0b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:42.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.355+0000 7f5e43bdc640 1 -- 172.21.15.122:0/274499595 shutdown_connections 2024-03-16T20:45:42.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.355+0000 7f5e43bdc640 1 -- 172.21.15.122:0/274499595 wait complete. 2024-03-16T20:45:42.374 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-16T20:45:42.374 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-16T20:45:42.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.479+0000 7f4539f63640 1 Processor -- start 2024-03-16T20:45:42.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.479+0000 7f4539f63640 1 -- start start 2024-03-16T20:45:42.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.479+0000 7f4539f63640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f453409ae30 0x7f45340af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:42.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.479+0000 7f4539f63640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f45340b4020 0x7f45340b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:42.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.479+0000 7f4539f63640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45340b7820 0x7f45340b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:42.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.479+0000 7f4539f63640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45340b38e0 con 0x7f453409ae30 2024-03-16T20:45:42.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.479+0000 7f4539f63640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453409b210 con 0x7f45340b4020 2024-03-16T20:45:42.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.479+0000 7f4539f63640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45340b4fb0 con 0x7f45340b7820 2024-03-16T20:45:42.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.479+0000 7f45337fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f453409ae30 0x7f45340af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:42.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.479+0000 7f45337fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f453409ae30 0x7f45340af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42284/0 (socket says 172.21.15.122:42284) 2024-03-16T20:45:42.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f45337fe640 1 -- 172.21.15.122:0/883938394 learned_addr learned my addr 172.21.15.122:0/883938394 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:42.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f45337fe640 1 -- 172.21.15.122:0/883938394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45340b7820 msgr2=0x7f45340b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:42.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f45337fe640 1 --2- 172.21.15.122:0/883938394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45340b7820 0x7f45340b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f45337fe640 1 -- 172.21.15.122:0/883938394 >> v2:172.21.15.195:3300/0 conn(0x7f45340b4020 msgr2=0x7f45340b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:42.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f45337fe640 1 --2- 172.21.15.122:0/883938394 >> v2:172.21.15.195:3300/0 conn(0x7f45340b4020 0x7f45340b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f45337fe640 1 -- 172.21.15.122:0/883938394 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45340b3290 con 0x7f453409ae30 2024-03-16T20:45:42.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f45337fe640 1 --2- 172.21.15.122:0/883938394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f453409ae30 0x7f45340af080 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f453406e490 tx=0x7f452401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7817541fa6cc74ae server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:42.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f45327fc640 1 -- 172.21.15.122:0/883938394 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45240ab670 con 0x7f453409ae30 2024-03-16T20:45:42.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f45327fc640 1 -- 172.21.15.122:0/883938394 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45240ab7d0 con 0x7f453409ae30 2024-03-16T20:45:42.488 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f45327fc640 1 -- 172.21.15.122:0/883938394 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45240abb60 con 0x7f453409ae30 2024-03-16T20:45:42.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f4539f63640 1 -- 172.21.15.122:0/883938394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f453409ae30 msgr2=0x7f45340af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:42.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f4539f63640 1 --2- 172.21.15.122:0/883938394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f453409ae30 0x7f45340af080 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f453406e490 tx=0x7f452401c490 comp rx=0 tx=0).stop 2024-03-16T20:45:42.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f4539f63640 1 -- 172.21.15.122:0/883938394 shutdown_connections 2024-03-16T20:45:42.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f4539f63640 1 --2- 172.21.15.122:0/883938394 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45340b7820 0x7f45340b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f4539f63640 1 --2- 172.21.15.122:0/883938394 >> v2:172.21.15.195:3300/0 conn(0x7f45340b4020 0x7f45340b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f4539f63640 1 --2- 172.21.15.122:0/883938394 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f453409ae30 0x7f45340af080 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f4539f63640 1 -- 172.21.15.122:0/883938394 >> 172.21.15.122:0/883938394 conn(0x7f45340b8d10 msgr2=0x7f453409a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:42.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f4539f63640 1 -- 172.21.15.122:0/883938394 shutdown_connections 2024-03-16T20:45:42.490 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.483+0000 7f4539f63640 1 -- 172.21.15.122:0/883938394 wait complete. 2024-03-16T20:45:42.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4539f63640 1 Processor -- start 2024-03-16T20:45:42.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4539f63640 1 -- start start 2024-03-16T20:45:42.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4539f63640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f453409ae30 0x7f4534099c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:42.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4539f63640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45340b4020 0x7f45340b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:42.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4539f63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45340b7820 0x7f453413b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:42.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4539f63640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45340b0320 con 0x7f45340b7820 2024-03-16T20:45:42.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4539f63640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45340b0dd0 con 0x7f453409ae30 2024-03-16T20:45:42.492 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4539f63640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45340b1b40 con 0x7f45340b4020 2024-03-16T20:45:42.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4533fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45340b7820 0x7f453413b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:42.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4533fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45340b7820 0x7f453413b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42296/0 (socket says 172.21.15.122:42296) 2024-03-16T20:45:42.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4533fff640 1 -- 172.21.15.122:0/951071129 learned_addr learned my addr 172.21.15.122:0/951071129 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:42.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f45337fe640 1 --2- 172.21.15.122:0/951071129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f453409ae30 0x7f4534099c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:42.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4533fff640 1 -- 172.21.15.122:0/951071129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45340b4020 msgr2=0x7f45340b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:42.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4533fff640 1 --2- 172.21.15.122:0/951071129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45340b4020 0x7f45340b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4533fff640 1 -- 172.21.15.122:0/951071129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f453409ae30 msgr2=0x7f4534099c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:42.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4533fff640 1 --2- 172.21.15.122:0/951071129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f453409ae30 0x7f4534099c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4533fff640 1 -- 172.21.15.122:0/951071129 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45240094d0 con 0x7f45340b7820 2024-03-16T20:45:42.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4533fff640 1 --2- 172.21.15.122:0/951071129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45340b7820 0x7f453413b7c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f452c00b970 tx=0x7f452c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:42.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f451bfff640 1 -- 172.21.15.122:0/951071129 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c00cb40 con 0x7f45340b7820 2024-03-16T20:45:42.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f451bfff640 1 -- 172.21.15.122:0/951071129 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c00ccc0 con 0x7f45340b7820 2024-03-16T20:45:42.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4539f63640 1 -- 172.21.15.122:0/951071129 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45340b1e20 con 0x7f45340b7820 2024-03-16T20:45:42.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f451bfff640 1 -- 172.21.15.122:0/951071129 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c014680 con 0x7f45340b7820 2024-03-16T20:45:42.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4539f63640 1 -- 172.21.15.122:0/951071129 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45340b08b0 con 0x7f45340b7820 2024-03-16T20:45:42.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f4539f63640 1 -- 172.21.15.122:0/951071129 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45340b0ad0 con 0x7f45340b7820 2024-03-16T20:45:42.494 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f451bfff640 1 -- 172.21.15.122:0/951071129 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f452c01d3b0 con 0x7f45340b7820 2024-03-16T20:45:42.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f451bfff640 1 --2- 172.21.15.122:0/951071129 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4520066e40 0x7f4520069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:42.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f451bfff640 1 -- 172.21.15.122:0/951071129 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f452c0845c0 con 0x7f45340b7820 2024-03-16T20:45:42.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.487+0000 7f45337fe640 1 --2- 172.21.15.122:0/951071129 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4520066e40 0x7f4520069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:42.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.491+0000 7f45337fe640 1 --2- 172.21.15.122:0/951071129 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4520066e40 0x7f4520069300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f45240af000 tx=0x7f45240a7000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:42.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.491+0000 7f451bfff640 1 -- 172.21.15.122:0/951071129 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f452c051790 con 0x7f45340b7820 2024-03-16T20:45:42.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.639+0000 7f4539f63640 1 -- 172.21.15.122:0/951071129 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4534074bb0 con 0x7f45340b7820 2024-03-16T20:45:42.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.903+0000 7f451bfff640 1 -- 172.21.15.122:0/951071129 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v27) v1 ==== 138+0+0 (secure 0 0 0) 0x7f452c0190b0 con 0x7f45340b7820 2024-03-16T20:45:42.909 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' created 2024-03-16T20:45:42.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.907+0000 7f4539f63640 1 -- 172.21.15.122:0/951071129 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4520066e40 msgr2=0x7f4520069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:42.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.907+0000 7f4539f63640 1 --2- 172.21.15.122:0/951071129 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4520066e40 0x7f4520069300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f45240af000 tx=0x7f45240a7000 comp rx=0 tx=0).stop 2024-03-16T20:45:42.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.907+0000 7f4539f63640 1 -- 172.21.15.122:0/951071129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45340b7820 msgr2=0x7f453413b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:42.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.907+0000 7f4539f63640 1 --2- 172.21.15.122:0/951071129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45340b7820 0x7f453413b7c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f452c00b970 tx=0x7f452c00be30 comp rx=0 tx=0).stop 2024-03-16T20:45:42.912 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.907+0000 7f4539f63640 1 -- 172.21.15.122:0/951071129 shutdown_connections 2024-03-16T20:45:42.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.907+0000 7f4539f63640 1 --2- 172.21.15.122:0/951071129 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4520066e40 0x7f4520069300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.907+0000 7f4539f63640 1 --2- 172.21.15.122:0/951071129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45340b7820 0x7f453413b7c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.907+0000 7f4539f63640 1 --2- 172.21.15.122:0/951071129 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45340b4020 0x7f45340b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.907+0000 7f4539f63640 1 --2- 172.21.15.122:0/951071129 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f453409ae30 0x7f4534099c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:42.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.907+0000 7f4539f63640 1 -- 172.21.15.122:0/951071129 >> 172.21.15.122:0/951071129 conn(0x7f45340b8d10 msgr2=0x7f453406c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:42.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.907+0000 7f4539f63640 1 -- 172.21.15.122:0/951071129 shutdown_connections 2024-03-16T20:45:42.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:42.907+0000 7f4539f63640 1 -- 172.21.15.122:0/951071129 wait complete. 2024-03-16T20:45:42.936 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-16T20:45:43.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.063+0000 7fe7c2007640 1 Processor -- start 2024-03-16T20:45:43.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.063+0000 7fe7c2007640 1 -- start start 2024-03-16T20:45:43.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.063+0000 7fe7c2007640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7bc09ae30 0x7fe7bc0b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:43.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.063+0000 7fe7c2007640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe7bc1429f0 0x7fe7bc0b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:43.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.063+0000 7fe7c2007640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc0b3fa0 con 0x7fe7bc0b7ba0 2024-03-16T20:45:43.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.063+0000 7fe7c2007640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7bc0b50a0 con 0x7fe7bc1429f0 2024-03-16T20:45:43.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.063+0000 7fe7c2007640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0b8560 con 0x7fe7bc09ae30 2024-03-16T20:45:43.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.063+0000 7fe7bbfff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe7bc1429f0 0x7fe7bc0b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:43.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.063+0000 7fe7baffd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fe7bc0b7ba0 0x7fe7bc1422e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39502/0 (socket says 172.21.15.122:39502) 2024-03-16T20:45:43.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.063+0000 7fe7baffd640 1 -- 172.21.15.122:0/4043531654 learned_addr learned my addr 172.21.15.122:0/4043531654 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:43.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.063+0000 7fe7bb7fe640 1 --2- 172.21.15.122:0/4043531654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7bc09ae30 0x7fe7bc0b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:43.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.063+0000 7fe7bbfff640 1 -- 172.21.15.122:0/4043531654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7bc09ae30 msgr2=0x7fe7bc0b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:43.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.063+0000 7fe7bbfff640 1 --2- 172.21.15.122:0/4043531654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7bc09ae30 0x7fe7bc0b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:43.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.063+0000 7fe7bbfff640 1 -- 172.21.15.122:0/4043531654 >> v1:172.21.15.122:6789/0 conn(0x7fe7bc0b7ba0 legacy=0x7fe7bc1422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:43.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.063+0000 7fe7bbfff640 1 -- 172.21.15.122:0/4043531654 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc09b210 con 0x7fe7bc1429f0 2024-03-16T20:45:43.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.063+0000 7fe7ba7fc640 1 -- 172.21.15.122:0/4043531654 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941663612 0 0) 0x7fe7bc0b3fa0 con 0x7fe7bc0b7ba0 2024-03-16T20:45:43.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.063+0000 7fe7bbfff640 1 --2- 172.21.15.122:0/4043531654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7bc1429f0 0x7fe7bc0b8020 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009d90 tx=0x7fe7b401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb1cb11fc4f4743 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:43.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7ba7fc640 1 -- 172.21.15.122:0/4043531654 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40a5a30 con 0x7fe7bc1429f0 2024-03-16T20:45:43.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7ba7fc640 1 -- 172.21.15.122:0/4043531654 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b40a5b90 con 0x7fe7bc1429f0 2024-03-16T20:45:43.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7ba7fc640 1 -- 172.21.15.122:0/4043531654 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40ad440 con 0x7fe7bc1429f0 2024-03-16T20:45:43.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 -- 172.21.15.122:0/4043531654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7bc1429f0 msgr2=0x7fe7bc0b8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:43.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 --2- 172.21.15.122:0/4043531654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7bc1429f0 0x7fe7bc0b8020 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4009d90 tx=0x7fe7b401c970 comp rx=0 tx=0).stop 2024-03-16T20:45:43.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 -- 172.21.15.122:0/4043531654 shutdown_connections 2024-03-16T20:45:43.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 --2- 172.21.15.122:0/4043531654 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7bc1429f0 0x7fe7bc0b8020 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:43.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 --2- 172.21.15.122:0/4043531654 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7bc09ae30 0x7fe7bc0b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:43.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 -- 172.21.15.122:0/4043531654 >> 172.21.15.122:0/4043531654 conn(0x7fe7bc0b8d10 msgr2=0x7fe7bc09a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:43.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 -- 172.21.15.122:0/4043531654 shutdown_connections 2024-03-16T20:45:43.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 -- 172.21.15.122:0/4043531654 wait complete. 2024-03-16T20:45:43.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 Processor -- start 2024-03-16T20:45:43.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 -- start start 2024-03-16T20:45:43.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7bc09ae30 0x7fe7bc070370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:43.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7bc0b7ba0 0x7fe7bc140e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:43.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7bc1429f0 0x7fe7bc0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:43.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0709c0 con 0x7fe7bc1429f0 2024-03-16T20:45:43.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc072490 con 0x7fe7bc0b7ba0 2024-03-16T20:45:43.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc071e40 con 0x7fe7bc09ae30 2024-03-16T20:45:43.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7bbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7bc1429f0 0x7fe7bc0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:43.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7bbfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7bc1429f0 0x7fe7bc0afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42314/0 (socket says 172.21.15.122:42314) 2024-03-16T20:45:43.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7bbfff640 1 -- 172.21.15.122:0/2503887534 learned_addr learned my addr 172.21.15.122:0/2503887534 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:43.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7baffd640 1 --2- 172.21.15.122:0/2503887534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7bc0b7ba0 0x7fe7bc140e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:43.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7bbfff640 1 -- 172.21.15.122:0/2503887534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7bc09ae30 msgr2=0x7fe7bc070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:43.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7bbfff640 1 --2- 172.21.15.122:0/2503887534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7bc09ae30 0x7fe7bc070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:43.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7bbfff640 1 -- 172.21.15.122:0/2503887534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7bc0b7ba0 msgr2=0x7fe7bc140e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:43.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7bbfff640 1 --2- 172.21.15.122:0/2503887534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7bc0b7ba0 0x7fe7bc140e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:43.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7bbfff640 1 -- 172.21.15.122:0/2503887534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b4009a70 con 0x7fe7bc1429f0 2024-03-16T20:45:43.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7baffd640 1 --2- 172.21.15.122:0/2503887534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7bc0b7ba0 0x7fe7bc140e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:43.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7bbfff640 1 --2- 172.21.15.122:0/2503887534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7bc1429f0 0x7fe7bc0afac0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe7b401b020 tx=0x7fe7b4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:43.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7a3fff640 1 -- 172.21.15.122:0/2503887534 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40addc0 con 0x7fe7bc1429f0 2024-03-16T20:45:43.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7a3fff640 1 -- 172.21.15.122:0/2503887534 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b40b9450 con 0x7fe7bc1429f0 2024-03-16T20:45:43.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 -- 172.21.15.122:0/2503887534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7bc072120 con 0x7fe7bc1429f0 2024-03-16T20:45:43.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7a3fff640 1 -- 172.21.15.122:0/2503887534 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40b97e0 con 0x7fe7bc1429f0 2024-03-16T20:45:43.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.067+0000 7fe7c2007640 1 -- 172.21.15.122:0/2503887534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc072920 con 0x7fe7bc1429f0 2024-03-16T20:45:43.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.071+0000 7fe7c2007640 1 -- 172.21.15.122:0/2503887534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7bc0b2c10 con 0x7fe7bc1429f0 2024-03-16T20:45:43.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.071+0000 7fe7a3fff640 1 -- 172.21.15.122:0/2503887534 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe7b40b9940 con 0x7fe7bc1429f0 2024-03-16T20:45:43.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.071+0000 7fe7a3fff640 1 --2- 172.21.15.122:0/2503887534 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe7a8066e40 0x7fe7a8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:43.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.071+0000 7fe7a3fff640 1 -- 172.21.15.122:0/2503887534 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fe7b4120a10 con 0x7fe7bc1429f0 2024-03-16T20:45:43.076 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.071+0000 7fe7bb7fe640 1 --2- 172.21.15.122:0/2503887534 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe7a8066e40 0x7fe7a8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:43.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.071+0000 7fe7bb7fe640 1 --2- 172.21.15.122:0/2503887534 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe7a8066e40 0x7fe7a8069300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00a8b0 tx=0x7fe7ac008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:43.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.075+0000 7fe7a3fff640 1 -- 172.21.15.122:0/2503887534 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe7b40edac0 con 0x7fe7bc1429f0 2024-03-16T20:45:43.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.231+0000 7fe7c2007640 1 -- 172.21.15.122:0/2503887534 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe7bc074b90 con 0x7fe7bc1429f0 2024-03-16T20:45:43.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.907+0000 7fe7a3fff640 1 -- 172.21.15.122:0/2503887534 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v28) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe7b40b20d0 con 0x7fe7bc1429f0 2024-03-16T20:45:43.913 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' created 2024-03-16T20:45:43.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.911+0000 7fe7c2007640 1 -- 172.21.15.122:0/2503887534 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe7a8066e40 msgr2=0x7fe7a8069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:43.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.911+0000 7fe7c2007640 1 --2- 172.21.15.122:0/2503887534 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe7a8066e40 0x7fe7a8069300 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00a8b0 tx=0x7fe7ac008040 comp rx=0 tx=0).stop 2024-03-16T20:45:43.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.911+0000 7fe7c2007640 1 -- 172.21.15.122:0/2503887534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7bc1429f0 msgr2=0x7fe7bc0afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:43.918 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.911+0000 7fe7c2007640 1 --2- 172.21.15.122:0/2503887534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7bc1429f0 0x7fe7bc0afac0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe7b401b020 tx=0x7fe7b4019710 comp rx=0 tx=0).stop 2024-03-16T20:45:43.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.911+0000 7fe7c2007640 1 -- 172.21.15.122:0/2503887534 shutdown_connections 2024-03-16T20:45:43.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.911+0000 7fe7c2007640 1 --2- 172.21.15.122:0/2503887534 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe7a8066e40 0x7fe7a8069300 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:43.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.911+0000 7fe7c2007640 1 --2- 172.21.15.122:0/2503887534 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7bc1429f0 0x7fe7bc0afac0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:43.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.911+0000 7fe7c2007640 1 --2- 172.21.15.122:0/2503887534 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7bc0b7ba0 0x7fe7bc140e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:43.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.911+0000 7fe7c2007640 1 --2- 172.21.15.122:0/2503887534 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7bc09ae30 0x7fe7bc070370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:43.919 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.911+0000 7fe7c2007640 1 -- 172.21.15.122:0/2503887534 >> 172.21.15.122:0/2503887534 conn(0x7fe7bc0b8d10 msgr2=0x7fe7bc1420b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:43.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.915+0000 7fe7c2007640 1 -- 172.21.15.122:0/2503887534 shutdown_connections 2024-03-16T20:45:43.920 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:43.915+0000 7fe7c2007640 1 -- 172.21.15.122:0/2503887534 wait complete. 2024-03-16T20:45:43.955 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-16T20:45:44.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.067+0000 7ff68826f640 1 Processor -- start 2024-03-16T20:45:44.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.067+0000 7ff68826f640 1 -- start start 2024-03-16T20:45:44.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.067+0000 7ff68826f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff6800b7540 0x7ff6800b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:44.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.067+0000 7ff68826f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff6800b7d40 0x7ff68013ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:44.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.067+0000 7ff68826f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6800b6740 con 0x7ff68013f460 2024-03-16T20:45:44.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.067+0000 7ff68826f640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6800b23d0 con 0x7ff6800b7d40 2024-03-16T20:45:44.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.067+0000 7ff68826f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6800b7110 con 0x7ff6800b7540 2024-03-16T20:45:44.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.071+0000 7ff68580a640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff6800b7d40 0x7ff68013ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:44.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.071+0000 7ff68580a640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff6800b7d40 0x7ff68013ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42324/0 (socket says 172.21.15.122:42324) 2024-03-16T20:45:44.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.071+0000 7ff68580a640 1 -- 172.21.15.122:0/797951542 learned_addr learned my addr 172.21.15.122:0/797951542 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:44.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.071+0000 7ff68600b640 1 --2- 172.21.15.122:0/797951542 >> v2:172.21.15.195:3300/0 conn(0x7ff6800b7540 0x7ff6800b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:44.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.071+0000 7ff68580a640 1 -- 172.21.15.122:0/797951542 >> v2:172.21.15.195:3300/0 conn(0x7ff6800b7540 msgr2=0x7ff6800b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:44.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.071+0000 7ff68580a640 1 --2- 172.21.15.122:0/797951542 >> v2:172.21.15.195:3300/0 conn(0x7ff6800b7540 0x7ff6800b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:44.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.071+0000 7ff68580a640 1 -- 172.21.15.122:0/797951542 >> v1:172.21.15.195:6789/0 conn(0x7ff68013f460 legacy=0x7ff6800b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:44.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.071+0000 7ff68580a640 1 -- 172.21.15.122:0/797951542 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6800b7920 con 0x7ff6800b7d40 2024-03-16T20:45:44.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.071+0000 7ff68600b640 1 --2- 172.21.15.122:0/797951542 >> v2:172.21.15.195:3300/0 conn(0x7ff6800b7540 0x7ff6800b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:44.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.071+0000 7ff68580a640 1 --2- 172.21.15.122:0/797951542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6800b7d40 0x7ff68013ef20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff6800678c0 tx=0x7ff67001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a358907f8cfb00 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:44.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.071+0000 7ff685009640 1 -- 172.21.15.122:0/797951542 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6700aa610 con 0x7ff6800b7d40 2024-03-16T20:45:44.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.071+0000 7ff685009640 1 -- 172.21.15.122:0/797951542 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6700a6040 con 0x7ff6800b7d40 2024-03-16T20:45:44.077 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.071+0000 7ff685009640 1 -- 172.21.15.122:0/797951542 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6700aa990 con 0x7ff6800b7d40 2024-03-16T20:45:44.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 -- 172.21.15.122:0/797951542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6800b7d40 msgr2=0x7ff68013ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:44.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 --2- 172.21.15.122:0/797951542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6800b7d40 0x7ff68013ef20 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff6800678c0 tx=0x7ff67001c4b0 comp rx=0 tx=0).stop 2024-03-16T20:45:44.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 -- 172.21.15.122:0/797951542 shutdown_connections 2024-03-16T20:45:44.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 --2- 172.21.15.122:0/797951542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6800b7d40 0x7ff68013ef20 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:44.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 --2- 172.21.15.122:0/797951542 >> v2:172.21.15.195:3300/0 conn(0x7ff6800b7540 0x7ff6800b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:44.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 -- 172.21.15.122:0/797951542 >> 172.21.15.122:0/797951542 conn(0x7ff68006fc00 msgr2=0x7ff680070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:44.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 -- 172.21.15.122:0/797951542 shutdown_connections 2024-03-16T20:45:44.078 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 -- 172.21.15.122:0/797951542 wait complete. 2024-03-16T20:45:44.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 Processor -- start 2024-03-16T20:45:44.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 -- start start 2024-03-16T20:45:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6800b7540 0x7ff68013d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6800b7d40 0x7ff680141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff68013f460 0x7ff680141e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68013dc80 con 0x7ff6800b7d40 2024-03-16T20:45:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6801387b0 con 0x7ff68013f460 2024-03-16T20:45:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6801d7870 con 0x7ff6800b7540 2024-03-16T20:45:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68580a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6800b7d40 0x7ff680141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68580a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6800b7d40 0x7ff680141950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42332/0 (socket says 172.21.15.122:42332) 2024-03-16T20:45:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68580a640 1 -- 172.21.15.122:0/187589758 learned_addr learned my addr 172.21.15.122:0/187589758 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68600b640 1 --2- 172.21.15.122:0/187589758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6800b7540 0x7ff68013d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68680c640 1 --2- 172.21.15.122:0/187589758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff68013f460 0x7ff680141e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68580a640 1 -- 172.21.15.122:0/187589758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6800b7540 msgr2=0x7ff68013d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:44.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68580a640 1 --2- 172.21.15.122:0/187589758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6800b7540 0x7ff68013d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:44.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68580a640 1 -- 172.21.15.122:0/187589758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff68013f460 msgr2=0x7ff680141e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:44.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68580a640 1 --2- 172.21.15.122:0/187589758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff68013f460 0x7ff680141e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:44.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68580a640 1 -- 172.21.15.122:0/187589758 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6700094d0 con 0x7ff6800b7d40 2024-03-16T20:45:44.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68680c640 1 --2- 172.21.15.122:0/187589758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff68013f460 0x7ff680141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:44.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68580a640 1 --2- 172.21.15.122:0/187589758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6800b7d40 0x7ff680141950 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff67001a0e0 tx=0x7ff67001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:44.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff676ffd640 1 -- 172.21.15.122:0/187589758 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6700b3800 con 0x7ff6800b7d40 2024-03-16T20:45:44.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff676ffd640 1 -- 172.21.15.122:0/187589758 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6700b3980 con 0x7ff6800b7d40 2024-03-16T20:45:44.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 -- 172.21.15.122:0/187589758 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68013df30 con 0x7ff6800b7d40 2024-03-16T20:45:44.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff676ffd640 1 -- 172.21.15.122:0/187589758 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6700b3d30 con 0x7ff6800b7d40 2024-03-16T20:45:44.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 -- 172.21.15.122:0/187589758 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6801e1370 con 0x7ff6800b7d40 2024-03-16T20:45:44.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.075+0000 7ff68826f640 1 -- 172.21.15.122:0/187589758 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff680138a20 con 0x7ff6800b7d40 2024-03-16T20:45:44.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.079+0000 7ff676ffd640 1 -- 172.21.15.122:0/187589758 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7ff6700c6390 con 0x7ff6800b7d40 2024-03-16T20:45:44.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.079+0000 7ff676ffd640 1 --2- 172.21.15.122:0/187589758 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff664066e70 0x7ff664069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:44.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.079+0000 7ff676ffd640 1 -- 172.21.15.122:0/187589758 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5609+0+0 (secure 0 0 0) 0x7ff67011f450 con 0x7ff6800b7d40 2024-03-16T20:45:44.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.079+0000 7ff68600b640 1 --2- 172.21.15.122:0/187589758 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff664066e70 0x7ff664069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:44.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.079+0000 7ff68600b640 1 --2- 172.21.15.122:0/187589758 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff664066e70 0x7ff664069330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff678006fd0 tx=0x7ff678008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:44.087 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.083+0000 7ff676ffd640 1 -- 172.21.15.122:0/187589758 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff6700ec330 con 0x7ff6800b7d40 2024-03-16T20:45:44.280 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.275+0000 7ff68826f640 1 -- 172.21.15.122:0/187589758 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff68006e4e0 con 0x7ff6800b7d40 2024-03-16T20:45:44.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.991+0000 7ff676ffd640 1 -- 172.21.15.122:0/187589758 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff6700ef9f0 con 0x7ff6800b7d40 2024-03-16T20:45:44.994 INFO:teuthology.orchestra.run.smithi122.stderr:new fs with metadata pool 4 and data pool 5 2024-03-16T20:45:44.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.995+0000 7ff68826f640 1 -- 172.21.15.122:0/187589758 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff664066e70 msgr2=0x7ff664069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:44.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.995+0000 7ff68826f640 1 --2- 172.21.15.122:0/187589758 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff664066e70 0x7ff664069330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff678006fd0 tx=0x7ff678008040 comp rx=0 tx=0).stop 2024-03-16T20:45:44.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.995+0000 7ff68826f640 1 -- 172.21.15.122:0/187589758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6800b7d40 msgr2=0x7ff680141950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:44.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.995+0000 7ff68826f640 1 --2- 172.21.15.122:0/187589758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6800b7d40 0x7ff680141950 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff67001a0e0 tx=0x7ff67001acf0 comp rx=0 tx=0).stop 2024-03-16T20:45:44.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.995+0000 7ff68826f640 1 -- 172.21.15.122:0/187589758 shutdown_connections 2024-03-16T20:45:44.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.995+0000 7ff68826f640 1 --2- 172.21.15.122:0/187589758 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff664066e70 0x7ff664069330 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:44.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.995+0000 7ff68826f640 1 --2- 172.21.15.122:0/187589758 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff68013f460 0x7ff680141e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:44.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.995+0000 7ff68826f640 1 --2- 172.21.15.122:0/187589758 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff6800b7d40 0x7ff680141950 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:44.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.995+0000 7ff68826f640 1 --2- 172.21.15.122:0/187589758 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff6800b7540 0x7ff68013d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.995+0000 7ff68826f640 1 -- 172.21.15.122:0/187589758 >> 172.21.15.122:0/187589758 conn(0x7ff68006fc00 msgr2=0x7ff6800b8720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:45.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.995+0000 7ff68826f640 1 -- 172.21.15.122:0/187589758 shutdown_connections 2024-03-16T20:45:45.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:44.995+0000 7ff68826f640 1 -- 172.21.15.122:0/187589758 wait complete. 2024-03-16T20:45:45.028 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:45.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.139+0000 7f09a3673640 1 Processor -- start 2024-03-16T20:45:45.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.139+0000 7f09a3673640 1 -- start start 2024-03-16T20:45:45.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.139+0000 7f09a3673640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f099c0baca0 0x7f099c144e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.139+0000 7f09a3673640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f099c148ec0 0x7f099c0b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.139+0000 7f09a3673640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c13eb30 0x7f099c13efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.146 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.139+0000 7f09a3673640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099c0b06b0 con 0x7f099c0baca0 2024-03-16T20:45:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.139+0000 7f09a3673640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f099c0b1cd0 con 0x7f099c148ec0 2024-03-16T20:45:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.139+0000 7f09a3673640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c0bb080 con 0x7f099c13eb30 2024-03-16T20:45:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a140f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f099c0baca0 0x7f099c144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a140f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f099c0baca0 0x7f099c144e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42342/0 (socket says 172.21.15.122:42342) 2024-03-16T20:45:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a140f640 1 -- 172.21.15.122:0/3378033092 learned_addr learned my addr 172.21.15.122:0/3378033092 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a0c0e640 1 --2- 172.21.15.122:0/3378033092 >> v2:172.21.15.195:3300/0 conn(0x7f099c148ec0 0x7f099c0b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a1c10640 1 --2- 172.21.15.122:0/3378033092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c13eb30 0x7f099c13efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a140f640 1 -- 172.21.15.122:0/3378033092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c13eb30 msgr2=0x7f099c13efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a140f640 1 --2- 172.21.15.122:0/3378033092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c13eb30 0x7f099c13efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a140f640 1 -- 172.21.15.122:0/3378033092 >> v2:172.21.15.195:3300/0 conn(0x7f099c148ec0 msgr2=0x7f099c0b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a140f640 1 --2- 172.21.15.122:0/3378033092 >> v2:172.21.15.195:3300/0 conn(0x7f099c148ec0 0x7f099c0b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a140f640 1 -- 172.21.15.122:0/3378033092 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099c0b1300 con 0x7f099c0baca0 2024-03-16T20:45:45.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a140f640 1 --2- 172.21.15.122:0/3378033092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f099c0baca0 0x7f099c144e10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f099c06e650 tx=0x7f099801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe86fd2aa6bbb90 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:45.148 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f0993fff640 1 -- 172.21.15.122:0/3378033092 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09980ab600 con 0x7f099c0baca0 2024-03-16T20:45:45.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f0993fff640 1 -- 172.21.15.122:0/3378033092 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09980a7050 con 0x7f099c0baca0 2024-03-16T20:45:45.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f0993fff640 1 -- 172.21.15.122:0/3378033092 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09980ab980 con 0x7f099c0baca0 2024-03-16T20:45:45.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a3673640 1 -- 172.21.15.122:0/3378033092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f099c0baca0 msgr2=0x7f099c144e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a3673640 1 --2- 172.21.15.122:0/3378033092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f099c0baca0 0x7f099c144e10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f099c06e650 tx=0x7f099801c520 comp rx=0 tx=0).stop 2024-03-16T20:45:45.149 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a3673640 1 -- 172.21.15.122:0/3378033092 shutdown_connections 2024-03-16T20:45:45.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a3673640 1 --2- 172.21.15.122:0/3378033092 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c13eb30 0x7f099c13efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a3673640 1 --2- 172.21.15.122:0/3378033092 >> v2:172.21.15.195:3300/0 conn(0x7f099c148ec0 0x7f099c0b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a3673640 1 --2- 172.21.15.122:0/3378033092 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f099c0baca0 0x7f099c144e10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a3673640 1 -- 172.21.15.122:0/3378033092 >> 172.21.15.122:0/3378033092 conn(0x7f099c06ee40 msgr2=0x7f099c06f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:45.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a3673640 1 -- 172.21.15.122:0/3378033092 shutdown_connections 2024-03-16T20:45:45.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.143+0000 7f09a3673640 1 -- 172.21.15.122:0/3378033092 wait complete. 2024-03-16T20:45:45.150 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a3673640 1 Processor -- start 2024-03-16T20:45:45.151 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a3673640 1 -- start start 2024-03-16T20:45:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a3673640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f099c0baca0 0x7f099c1e1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a3673640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c13eb30 0x7f099c144030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a3673640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f099c148ec0 0x7f099c0b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a3673640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c1e1b90 con 0x7f099c0baca0 2024-03-16T20:45:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a3673640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f099c0b3940 con 0x7f099c148ec0 2024-03-16T20:45:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a3673640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f099c140240 con 0x7f099c13eb30 2024-03-16T20:45:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a140f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f099c0baca0 0x7f099c1e1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.152 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a140f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f099c0baca0 0x7f099c1e1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42348/0 (socket says 172.21.15.122:42348) 2024-03-16T20:45:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a140f640 1 -- 172.21.15.122:0/714064914 learned_addr learned my addr 172.21.15.122:0/714064914 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a140f640 1 -- 172.21.15.122:0/714064914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c13eb30 msgr2=0x7f099c144030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a140f640 1 --2- 172.21.15.122:0/714064914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c13eb30 0x7f099c144030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a140f640 1 -- 172.21.15.122:0/714064914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f099c148ec0 msgr2=0x7f099c0b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a140f640 1 --2- 172.21.15.122:0/714064914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f099c148ec0 0x7f099c0b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a140f640 1 -- 172.21.15.122:0/714064914 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09980094d0 con 0x7f099c0baca0 2024-03-16T20:45:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a140f640 1 --2- 172.21.15.122:0/714064914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f099c0baca0 0x7f099c1e1fb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f09980ae040 tx=0x7f09980192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f0991ffb640 1 -- 172.21.15.122:0/714064914 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09980abdc0 con 0x7f099c0baca0 2024-03-16T20:45:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f0991ffb640 1 -- 172.21.15.122:0/714064914 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09980b4440 con 0x7f099c0baca0 2024-03-16T20:45:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a3673640 1 -- 172.21.15.122:0/714064914 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f099c1e14d0 con 0x7f099c0baca0 2024-03-16T20:45:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f0991ffb640 1 -- 172.21.15.122:0/714064914 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09980b47f0 con 0x7f099c0baca0 2024-03-16T20:45:45.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a3673640 1 -- 172.21.15.122:0/714064914 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f099c13fd60 con 0x7f099c0baca0 2024-03-16T20:45:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f09a3673640 1 -- 172.21.15.122:0/714064914 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f099c0677b0 con 0x7f099c0baca0 2024-03-16T20:45:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f0991ffb640 1 -- 172.21.15.122:0/714064914 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f09980b2070 con 0x7f099c0baca0 2024-03-16T20:45:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.147+0000 7f0991ffb640 1 --2- 172.21.15.122:0/714064914 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0980066e40 0x7f0980069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.151+0000 7f0991ffb640 1 -- 172.21.15.122:0/714064914 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f099811e7f0 con 0x7f099c0baca0 2024-03-16T20:45:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.151+0000 7f09a0c0e640 1 --2- 172.21.15.122:0/714064914 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0980066e40 0x7f0980069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.155 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.151+0000 7f09a0c0e640 1 --2- 172.21.15.122:0/714064914 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0980066e40 0x7f0980069300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f098c006fd0 tx=0x7f098c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:45.158 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.155+0000 7f0991ffb640 1 -- 172.21.15.122:0/714064914 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f09980eb670 con 0x7f099c0baca0 2024-03-16T20:45:45.318 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.315+0000 7f09a3673640 1 -- 172.21.15.122:0/714064914 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f099c1e1cd0 con 0x7f099c0baca0 2024-03-16T20:45:45.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.315+0000 7f0991ffb640 1 -- 172.21.15.122:0/714064914 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+16718 (secure 0 0 0) 0x7f099c1e1cd0 con 0x7f099c0baca0 2024-03-16T20:45:45.321 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:45.321 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":29,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:44.279688+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:42.678541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:43.274469+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:45.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.319+0000 7f09a3673640 1 -- 172.21.15.122:0/714064914 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0980066e40 msgr2=0x7f0980069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.319+0000 7f09a3673640 1 --2- 172.21.15.122:0/714064914 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0980066e40 0x7f0980069300 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f098c006fd0 tx=0x7f098c008040 comp rx=0 tx=0).stop 2024-03-16T20:45:45.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.319+0000 7f09a3673640 1 -- 172.21.15.122:0/714064914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f099c0baca0 msgr2=0x7f099c1e1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.319+0000 7f09a3673640 1 --2- 172.21.15.122:0/714064914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f099c0baca0 0x7f099c1e1fb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f09980ae040 tx=0x7f09980192c0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.319+0000 7f09a3673640 1 -- 172.21.15.122:0/714064914 shutdown_connections 2024-03-16T20:45:45.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.319+0000 7f09a3673640 1 --2- 172.21.15.122:0/714064914 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0980066e40 0x7f0980069300 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.319+0000 7f09a3673640 1 --2- 172.21.15.122:0/714064914 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f099c148ec0 0x7f099c0b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.319+0000 7f09a3673640 1 --2- 172.21.15.122:0/714064914 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f099c13eb30 0x7f099c144030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.319+0000 7f09a3673640 1 --2- 172.21.15.122:0/714064914 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f099c0baca0 0x7f099c1e1fb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.319+0000 7f09a3673640 1 -- 172.21.15.122:0/714064914 >> 172.21.15.122:0/714064914 conn(0x7f099c06ee40 msgr2=0x7f099c070530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:45.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.319+0000 7f09a3673640 1 -- 172.21.15.122:0/714064914 shutdown_connections 2024-03-16T20:45:45.324 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.319+0000 7f09a3673640 1 -- 172.21.15.122:0/714064914 wait complete. 2024-03-16T20:45:45.340 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:45.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.455+0000 7f05438eb640 1 Processor -- start 2024-03-16T20:45:45.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.455+0000 7f05438eb640 1 -- start start 2024-03-16T20:45:45.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.455+0000 7f05438eb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f053c147e30 0x7f053c0b8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.455+0000 7f05438eb640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f053c06d8d0 con 0x7f053c0b7990 2024-03-16T20:45:45.461 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.455+0000 7f05438eb640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f053c147350 con 0x7f053c0b75b0 2024-03-16T20:45:45.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.455+0000 7f05438eb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053c147160 con 0x7f053c147e30 2024-03-16T20:45:45.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.455+0000 7f0540e86640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f053c0b7990 0x7f053c06d1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39518/0 (socket says 172.21.15.122:39518) 2024-03-16T20:45:45.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.455+0000 7f0540e86640 1 -- 172.21.15.122:0/3244344973 learned_addr learned my addr 172.21.15.122:0/3244344973 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:45.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0541e88640 1 --2- 172.21.15.122:0/3244344973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f053c147e30 0x7f053c0b8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 -- 172.21.15.122:0/3244344973 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4158745387 0 0) 0x7f053c06d8d0 con 0x7f053c0b7990 2024-03-16T20:45:45.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 -- 172.21.15.122:0/3244344973 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0528003530 con 0x7f053c0b7990 2024-03-16T20:45:45.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 -- 172.21.15.122:0/3244344973 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2386432023 0 0) 0x7f0528003530 con 0x7f053c0b7990 2024-03-16T20:45:45.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 -- 172.21.15.122:0/3244344973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f053c147e30 msgr2=0x7f053c0b8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:45.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 --2- 172.21.15.122:0/3244344973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f053c147e30 0x7f053c0b8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 -- 172.21.15.122:0/3244344973 >> v1:172.21.15.195:6789/0 conn(0x7f053c0b75b0 legacy=0x7f053c0b7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 -- 172.21.15.122:0/3244344973 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053c147630 con 0x7f053c0b7990 2024-03-16T20:45:45.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 -- 172.21.15.122:0/3244344973 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f052c001680 con 0x7f053c0b7990 2024-03-16T20:45:45.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 -- 172.21.15.122:0/3244344973 >> v1:172.21.15.122:6789/0 conn(0x7f053c0b7990 legacy=0x7f053c06d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 --2- 172.21.15.122:0/3244344973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05280039b0 0x7f0528003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 --2- 172.21.15.122:0/3244344973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05280047d0 0x7f0528004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 --2- 172.21.15.122:0/3244344973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f053c147e30 0x7f05280051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 -- 172.21.15.122:0/3244344973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052800ba90 con 0x7f05280047d0 2024-03-16T20:45:45.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 -- 172.21.15.122:0/3244344973 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052800bc00 con 0x7f05280039b0 2024-03-16T20:45:45.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 -- 172.21.15.122:0/3244344973 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f052800bdc0 con 0x7f053c147e30 2024-03-16T20:45:45.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0540e86640 1 --2- 172.21.15.122:0/3244344973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05280047d0 0x7f0528004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0541687640 1 --2- 172.21.15.122:0/3244344973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05280039b0 0x7f0528003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0541e88640 1 --2- 172.21.15.122:0/3244344973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f053c147e30 0x7f05280051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0540e86640 1 -- 172.21.15.122:0/3244344973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f053c147e30 msgr2=0x7f05280051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:45.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0540e86640 1 --2- 172.21.15.122:0/3244344973 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f053c147e30 0x7f05280051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0540e86640 1 -- 172.21.15.122:0/3244344973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05280039b0 msgr2=0x7f0528003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:45.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0540e86640 1 --2- 172.21.15.122:0/3244344973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05280039b0 0x7f0528003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0540e86640 1 -- 172.21.15.122:0/3244344973 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0528003530 con 0x7f05280047d0 2024-03-16T20:45:45.465 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0541687640 1 --2- 172.21.15.122:0/3244344973 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f05280039b0 0x7f0528003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:45.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0540e86640 1 --2- 172.21.15.122:0/3244344973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05280047d0 0x7f0528004c50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f052c00c210 tx=0x7f052c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224aac459fbe0310 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:45.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 -- 172.21.15.122:0/3244344973 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c005ae0 con 0x7f05280047d0 2024-03-16T20:45:45.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 -- 172.21.15.122:0/3244344973 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c005c60 con 0x7f05280047d0 2024-03-16T20:45:45.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0533fff640 1 -- 172.21.15.122:0/3244344973 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c007070 con 0x7f05280047d0 2024-03-16T20:45:45.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f05438eb640 1 -- 172.21.15.122:0/3244344973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05280047d0 msgr2=0x7f0528004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f05438eb640 1 --2- 172.21.15.122:0/3244344973 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05280047d0 0x7f0528004c50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f052c00c210 tx=0x7f052c01cc40 comp rx=0 tx=0).stop 2024-03-16T20:45:45.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f0541e88640 1 -- 172.21.15.122:0/3244344973 reap_dead start 2024-03-16T20:45:45.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f05438eb640 1 -- 172.21.15.122:0/3244344973 shutdown_connections 2024-03-16T20:45:45.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f05438eb640 1 -- 172.21.15.122:0/3244344973 >> 172.21.15.122:0/3244344973 conn(0x7f053c0bcdd0 msgr2=0x7f053c0bd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:45.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f05438eb640 1 -- 172.21.15.122:0/3244344973 shutdown_connections 2024-03-16T20:45:45.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.459+0000 7f05438eb640 1 -- 172.21.15.122:0/3244344973 wait complete. 2024-03-16T20:45:45.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f05438eb640 1 Processor -- start 2024-03-16T20:45:45.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f05438eb640 1 -- start start 2024-03-16T20:45:45.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f05438eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05280047d0 0x7f053c143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f05438eb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f053c144780 0x7f053c1dfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f05438eb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f053c1e0090 0x7f053c138e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f05438eb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c144580 con 0x7f05280047d0 2024-03-16T20:45:45.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f05438eb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053c13b7c0 con 0x7f053c1e0090 2024-03-16T20:45:45.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f05438eb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f053c1442a0 con 0x7f053c144780 2024-03-16T20:45:45.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f0541687640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05280047d0 0x7f053c143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f0541687640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05280047d0 0x7f053c143180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42364/0 (socket says 172.21.15.122:42364) 2024-03-16T20:45:45.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f0541687640 1 -- 172.21.15.122:0/180521538 learned_addr learned my addr 172.21.15.122:0/180521538 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:45.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f0541e88640 1 --2- 172.21.15.122:0/180521538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f053c1e0090 0x7f053c138e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f0540e86640 1 --2- 172.21.15.122:0/180521538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f053c144780 0x7f053c1dfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f0541687640 1 -- 172.21.15.122:0/180521538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f053c144780 msgr2=0x7f053c1dfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f0541687640 1 --2- 172.21.15.122:0/180521538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f053c144780 0x7f053c1dfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f0541687640 1 -- 172.21.15.122:0/180521538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f053c1e0090 msgr2=0x7f053c138e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f0541687640 1 --2- 172.21.15.122:0/180521538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f053c1e0090 0x7f053c138e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f0541687640 1 -- 172.21.15.122:0/180521538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f053c13a930 con 0x7f05280047d0 2024-03-16T20:45:45.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f0541687640 1 --2- 172.21.15.122:0/180521538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05280047d0 0x7f053c143180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f053800d540 tx=0x7f053800da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:45.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f0531ffb640 1 -- 172.21.15.122:0/180521538 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0538005480 con 0x7f05280047d0 2024-03-16T20:45:45.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f05438eb640 1 -- 172.21.15.122:0/180521538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053c144c00 con 0x7f05280047d0 2024-03-16T20:45:45.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f0531ffb640 1 -- 172.21.15.122:0/180521538 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0538003620 con 0x7f05280047d0 2024-03-16T20:45:45.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f0531ffb640 1 -- 172.21.15.122:0/180521538 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05380039f0 con 0x7f05280047d0 2024-03-16T20:45:45.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f05438eb640 1 -- 172.21.15.122:0/180521538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f053c143ec0 con 0x7f05280047d0 2024-03-16T20:45:45.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.463+0000 7f05438eb640 1 -- 172.21.15.122:0/180521538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053c13aac0 con 0x7f05280047d0 2024-03-16T20:45:45.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.467+0000 7f0531ffb640 1 -- 172.21.15.122:0/180521538 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f0538003b50 con 0x7f05280047d0 2024-03-16T20:45:45.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.467+0000 7f0531ffb640 1 --2- 172.21.15.122:0/180521538 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0520066e70 0x7f0520069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.467+0000 7f0531ffb640 1 -- 172.21.15.122:0/180521538 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0538084780 con 0x7f05280047d0 2024-03-16T20:45:45.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.467+0000 7f0540e86640 1 --2- 172.21.15.122:0/180521538 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0520066e70 0x7f0520069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.467+0000 7f0540e86640 1 --2- 172.21.15.122:0/180521538 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0520066e70 0x7f0520069330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f052c0ab000 tx=0x7f052c0a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:45.474 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.471+0000 7f0531ffb640 1 -- 172.21.15.122:0/180521538 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0538051630 con 0x7f05280047d0 2024-03-16T20:45:45.634 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.631+0000 7f05438eb640 1 -- 172.21.15.122:0/180521538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f053c1440b0 con 0x7f05280047d0 2024-03-16T20:45:45.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.631+0000 7f0531ffb640 1 -- 172.21.15.122:0/180521538 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+16718 (secure 0 0 0) 0x7f0538001240 con 0x7f05280047d0 2024-03-16T20:45:45.636 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:45.636 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":29,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:44.279688+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:42.678541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:43.274469+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:45.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.635+0000 7f05438eb640 1 -- 172.21.15.122:0/180521538 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0520066e70 msgr2=0x7f0520069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.638 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.635+0000 7f05438eb640 1 --2- 172.21.15.122:0/180521538 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0520066e70 0x7f0520069330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f052c0ab000 tx=0x7f052c0a7070 comp rx=0 tx=0).stop 2024-03-16T20:45:45.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.635+0000 7f05438eb640 1 -- 172.21.15.122:0/180521538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05280047d0 msgr2=0x7f053c143180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.635+0000 7f05438eb640 1 --2- 172.21.15.122:0/180521538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05280047d0 0x7f053c143180 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f053800d540 tx=0x7f053800da00 comp rx=0 tx=0).stop 2024-03-16T20:45:45.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.635+0000 7f05438eb640 1 -- 172.21.15.122:0/180521538 shutdown_connections 2024-03-16T20:45:45.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.635+0000 7f05438eb640 1 --2- 172.21.15.122:0/180521538 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0520066e70 0x7f0520069330 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.635+0000 7f05438eb640 1 --2- 172.21.15.122:0/180521538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f053c1e0090 0x7f053c138e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.635+0000 7f05438eb640 1 --2- 172.21.15.122:0/180521538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f053c144780 0x7f053c1dfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.635+0000 7f05438eb640 1 --2- 172.21.15.122:0/180521538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05280047d0 0x7f053c143180 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.635+0000 7f05438eb640 1 -- 172.21.15.122:0/180521538 >> 172.21.15.122:0/180521538 conn(0x7f053c0bcdd0 msgr2=0x7f053c0a0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:45.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.635+0000 7f05438eb640 1 -- 172.21.15.122:0/180521538 shutdown_connections 2024-03-16T20:45:45.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.635+0000 7f05438eb640 1 -- 172.21.15.122:0/180521538 wait complete. 2024-03-16T20:45:45.656 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-16T20:45:45.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f171d30b640 1 Processor -- start 2024-03-16T20:45:45.774 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f171d30b640 1 -- start start 2024-03-16T20:45:45.776 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f171d30b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f17180b4020 0x7f17180b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f171d30b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17180b7820 0x7f17180b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f171d30b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17180b38e0 con 0x7f171809ae30 2024-03-16T20:45:45.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f171d30b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17180af790 con 0x7f17180b4020 2024-03-16T20:45:45.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f171d30b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171809b210 con 0x7f17180b7820 2024-03-16T20:45:45.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1716ffd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f171809ae30 0x7f17180af080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:39532/0 (socket says 172.21.15.122:39532) 2024-03-16T20:45:45.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1716ffd640 1 -- 172.21.15.122:0/3462945422 learned_addr learned my addr 172.21.15.122:0/3462945422 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:45.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f17167fc640 1 --2- 172.21.15.122:0/3462945422 >> v2:172.21.15.195:3300/0 conn(0x7f17180b4020 0x7f17180b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f17177fe640 1 --2- 172.21.15.122:0/3462945422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17180b7820 0x7f17180b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1715ffb640 1 -- 172.21.15.122:0/3462945422 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782653315 0 0) 0x7f17180b38e0 con 0x7f171809ae30 2024-03-16T20:45:45.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1715ffb640 1 -- 172.21.15.122:0/3462945422 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1700003530 con 0x7f171809ae30 2024-03-16T20:45:45.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1715ffb640 1 -- 172.21.15.122:0/3462945422 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 135584483 0 0) 0x7f1700003530 con 0x7f171809ae30 2024-03-16T20:45:45.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1715ffb640 1 -- 172.21.15.122:0/3462945422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17180b7820 msgr2=0x7f17180b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:45.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1715ffb640 1 --2- 172.21.15.122:0/3462945422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17180b7820 0x7f17180b6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1715ffb640 1 -- 172.21.15.122:0/3462945422 >> v2:172.21.15.195:3300/0 conn(0x7f17180b4020 msgr2=0x7f17180b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:45.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1715ffb640 1 --2- 172.21.15.122:0/3462945422 >> v2:172.21.15.195:3300/0 conn(0x7f17180b4020 0x7f17180b67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1715ffb640 1 -- 172.21.15.122:0/3462945422 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17180b4fb0 con 0x7f171809ae30 2024-03-16T20:45:45.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1715ffb640 1 -- 172.21.15.122:0/3462945422 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f170c001680 con 0x7f171809ae30 2024-03-16T20:45:45.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1715ffb640 1 -- 172.21.15.122:0/3462945422 >> v1:172.21.15.122:6789/0 conn(0x7f171809ae30 legacy=0x7f17180af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1715ffb640 1 --2- 172.21.15.122:0/3462945422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1700003980 0x7f1700003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1715ffb640 1 --2- 172.21.15.122:0/3462945422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17000047d0 0x7f1700004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1715ffb640 1 --2- 172.21.15.122:0/3462945422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17180b7820 0x7f1700005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1715ffb640 1 -- 172.21.15.122:0/3462945422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17180af790 con 0x7f17000047d0 2024-03-16T20:45:45.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1715ffb640 1 -- 172.21.15.122:0/3462945422 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171809b210 con 0x7f1700003980 2024-03-16T20:45:45.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1715ffb640 1 -- 172.21.15.122:0/3462945422 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170000b990 con 0x7f17180b7820 2024-03-16T20:45:45.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f17167fc640 1 --2- 172.21.15.122:0/3462945422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17000047d0 0x7f1700004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f1716ffd640 1 --2- 172.21.15.122:0/3462945422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1700003980 0x7f1700003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f17167fc640 1 -- 172.21.15.122:0/3462945422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17180b7820 msgr2=0x7f1700005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:45.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f17167fc640 1 --2- 172.21.15.122:0/3462945422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17180b7820 0x7f1700005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f17167fc640 1 -- 172.21.15.122:0/3462945422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1700003980 msgr2=0x7f1700003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:45.778 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f17167fc640 1 --2- 172.21.15.122:0/3462945422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1700003980 0x7f1700003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.779 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.771+0000 7f17167fc640 1 -- 172.21.15.122:0/3462945422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1700003530 con 0x7f17000047d0 2024-03-16T20:45:45.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f17167fc640 1 --2- 172.21.15.122:0/3462945422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17000047d0 0x7f1700004c50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1708009ae0 tx=0x7f170801ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278a85d6d0881f5c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:45.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f1715ffb640 1 -- 172.21.15.122:0/3462945422 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17080a5530 con 0x7f17000047d0 2024-03-16T20:45:45.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f1715ffb640 1 -- 172.21.15.122:0/3462945422 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17080a5690 con 0x7f17000047d0 2024-03-16T20:45:45.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f1715ffb640 1 -- 172.21.15.122:0/3462945422 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17080a5a40 con 0x7f17000047d0 2024-03-16T20:45:45.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f171d30b640 1 -- 172.21.15.122:0/3462945422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17000047d0 msgr2=0x7f1700004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f171d30b640 1 --2- 172.21.15.122:0/3462945422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17000047d0 0x7f1700004c50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f1708009ae0 tx=0x7f170801ca40 comp rx=0 tx=0).stop 2024-03-16T20:45:45.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f17177fe640 1 -- 172.21.15.122:0/3462945422 reap_dead start 2024-03-16T20:45:45.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f171d30b640 1 -- 172.21.15.122:0/3462945422 shutdown_connections 2024-03-16T20:45:45.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f171d30b640 1 -- 172.21.15.122:0/3462945422 >> 172.21.15.122:0/3462945422 conn(0x7f17180b8d10 msgr2=0x7f171809a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:45.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f171d30b640 1 -- 172.21.15.122:0/3462945422 shutdown_connections 2024-03-16T20:45:45.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f171d30b640 1 -- 172.21.15.122:0/3462945422 wait complete. 2024-03-16T20:45:45.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f171d30b640 1 Processor -- start 2024-03-16T20:45:45.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f171d30b640 1 -- start start 2024-03-16T20:45:45.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f171d30b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17000047d0 0x7f1718141180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f171d30b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17180b1750 0x7f17180b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f171d30b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17181e0f60 0x7f17181e1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f171d30b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17180b2110 con 0x7f17181e0f60 2024-03-16T20:45:45.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f171d30b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f171813f280 con 0x7f17000047d0 2024-03-16T20:45:45.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f171d30b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f171813d890 con 0x7f17180b1750 2024-03-16T20:45:45.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f17177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17181e0f60 0x7f17181e1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f17177fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17181e0f60 0x7f17181e1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42388/0 (socket says 172.21.15.122:42388) 2024-03-16T20:45:45.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f17177fe640 1 -- 172.21.15.122:0/2243880003 learned_addr learned my addr 172.21.15.122:0/2243880003 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:45.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f1716ffd640 1 --2- 172.21.15.122:0/2243880003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17000047d0 0x7f1718141180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f17167fc640 1 --2- 172.21.15.122:0/2243880003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17180b1750 0x7f17180b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f17177fe640 1 -- 172.21.15.122:0/2243880003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17180b1750 msgr2=0x7f17180b1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f17177fe640 1 --2- 172.21.15.122:0/2243880003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17180b1750 0x7f17180b1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f17177fe640 1 -- 172.21.15.122:0/2243880003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17000047d0 msgr2=0x7f1718141180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f17177fe640 1 --2- 172.21.15.122:0/2243880003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17000047d0 0x7f1718141180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f17177fe640 1 -- 172.21.15.122:0/2243880003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f171813c5b0 con 0x7f17181e0f60 2024-03-16T20:45:45.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f17177fe640 1 --2- 172.21.15.122:0/2243880003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17181e0f60 0x7f17181e1410 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f17180ada10 tx=0x7f171000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:45.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f16ff7fe640 1 -- 172.21.15.122:0/2243880003 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710010040 con 0x7f17181e0f60 2024-03-16T20:45:45.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f16ff7fe640 1 -- 172.21.15.122:0/2243880003 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171000bce0 con 0x7f17181e0f60 2024-03-16T20:45:45.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f16ff7fe640 1 -- 172.21.15.122:0/2243880003 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1710014640 con 0x7f17181e0f60 2024-03-16T20:45:45.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f171d30b640 1 -- 172.21.15.122:0/2243880003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171813c890 con 0x7f17181e0f60 2024-03-16T20:45:45.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.775+0000 7f171d30b640 1 -- 172.21.15.122:0/2243880003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1718072e00 con 0x7f17181e0f60 2024-03-16T20:45:45.784 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.779+0000 7f171d30b640 1 -- 172.21.15.122:0/2243880003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f171813ca80 con 0x7f17181e0f60 2024-03-16T20:45:45.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.779+0000 7f16ff7fe640 1 -- 172.21.15.122:0/2243880003 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f17100147a0 con 0x7f17181e0f60 2024-03-16T20:45:45.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.779+0000 7f16ff7fe640 1 --2- 172.21.15.122:0/2243880003 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1704066e40 0x7f1704069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:45.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.779+0000 7f16ff7fe640 1 -- 172.21.15.122:0/2243880003 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1710085170 con 0x7f17181e0f60 2024-03-16T20:45:45.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.779+0000 7f1716ffd640 1 --2- 172.21.15.122:0/2243880003 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1704066e40 0x7f1704069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:45.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.779+0000 7f1716ffd640 1 --2- 172.21.15.122:0/2243880003 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1704066e40 0x7f1704069300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f170c001060 tx=0x7f170c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:45.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.783+0000 7f16ff7fe640 1 -- 172.21.15.122:0/2243880003 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1710051f50 con 0x7f17181e0f60 2024-03-16T20:45:45.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.979+0000 7f171d30b640 1 -- 172.21.15.122:0/2243880003 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1718074bb0 con 0x7f17181e0f60 2024-03-16T20:45:45.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.991+0000 7f16ff7fe640 1 -- 172.21.15.122:0/2243880003 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1710055610 con 0x7f17181e0f60 2024-03-16T20:45:45.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.995+0000 7f171d30b640 1 -- 172.21.15.122:0/2243880003 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1704066e40 msgr2=0x7f1704069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.995+0000 7f171d30b640 1 --2- 172.21.15.122:0/2243880003 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1704066e40 0x7f1704069300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f170c001060 tx=0x7f170c009040 comp rx=0 tx=0).stop 2024-03-16T20:45:45.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.995+0000 7f171d30b640 1 -- 172.21.15.122:0/2243880003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17181e0f60 msgr2=0x7f17181e1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:45.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.995+0000 7f171d30b640 1 --2- 172.21.15.122:0/2243880003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17181e0f60 0x7f17181e1410 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f17180ada10 tx=0x7f171000b480 comp rx=0 tx=0).stop 2024-03-16T20:45:45.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.995+0000 7f171d30b640 1 -- 172.21.15.122:0/2243880003 shutdown_connections 2024-03-16T20:45:45.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.995+0000 7f171d30b640 1 --2- 172.21.15.122:0/2243880003 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f1704066e40 0x7f1704069300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.995+0000 7f171d30b640 1 --2- 172.21.15.122:0/2243880003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f17181e0f60 0x7f17181e1410 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.995+0000 7f171d30b640 1 --2- 172.21.15.122:0/2243880003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f17180b1750 0x7f17180b1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.995+0000 7f171d30b640 1 --2- 172.21.15.122:0/2243880003 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f17000047d0 0x7f1718141180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:45.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.995+0000 7f171d30b640 1 -- 172.21.15.122:0/2243880003 >> 172.21.15.122:0/2243880003 conn(0x7f17180b8d10 msgr2=0x7f171806c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:45.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.995+0000 7f171d30b640 1 -- 172.21.15.122:0/2243880003 shutdown_connections 2024-03-16T20:45:45.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:45.995+0000 7f171d30b640 1 -- 172.21.15.122:0/2243880003 wait complete. 2024-03-16T20:45:46.015 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-03-16T20:45:46.015 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-03-16T20:45:46.125 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.119+0000 7f50092ff640 1 Processor -- start 2024-03-16T20:45:46.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.119+0000 7f50092ff640 1 -- start start 2024-03-16T20:45:46.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.123+0000 7f50092ff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50040b7d40 0x7f50040b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:46.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.123+0000 7f50092ff640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5004148480 0x7f500414f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:46.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.123+0000 7f50092ff640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50040b6740 con 0x7f50040b7540 2024-03-16T20:45:46.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.123+0000 7f50092ff640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50040b23d0 con 0x7f5004148480 2024-03-16T20:45:46.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.123+0000 7f50092ff640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50040b7110 con 0x7f50040b7d40 2024-03-16T20:45:46.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.123+0000 7f4ffa7fc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50040b7d40 0x7f50040b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:46.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.123+0000 7f50037fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5004148480 0x7f500414f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:46.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.123+0000 7f5002ffd640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7f50040b7540 0x7f500413ef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.122:34862/0 (socket says 172.21.15.122:34862) 2024-03-16T20:45:46.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.123+0000 7f5002ffd640 1 -- 172.21.15.122:0/2021538508 learned_addr learned my addr 172.21.15.122:0/2021538508 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:46.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.123+0000 7f4ffa7fc640 1 -- 172.21.15.122:0/2021538508 >> v2:172.21.15.195:3300/0 conn(0x7f5004148480 msgr2=0x7f500414f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:46.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.123+0000 7f4ffa7fc640 1 --2- 172.21.15.122:0/2021538508 >> v2:172.21.15.195:3300/0 conn(0x7f5004148480 0x7f500414f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:46.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.123+0000 7f4ffa7fc640 1 -- 172.21.15.122:0/2021538508 >> v1:172.21.15.195:6789/0 conn(0x7f50040b7540 legacy=0x7f500413ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:46.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.123+0000 7f4ffa7fc640 1 -- 172.21.15.122:0/2021538508 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50040b7920 con 0x7f50040b7d40 2024-03-16T20:45:46.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.123+0000 7f50037fe640 1 --2- 172.21.15.122:0/2021538508 >> v2:172.21.15.195:3300/0 conn(0x7f5004148480 0x7f500414f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:45:46.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.127+0000 7f4ffa7fc640 1 --2- 172.21.15.122:0/2021538508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50040b7d40 0x7f50040b6bd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40097c0 tx=0x7f4ff401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61a0950fe4b3b15d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:46.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.127+0000 7f50027fc640 1 -- 172.21.15.122:0/2021538508 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff40ac640 con 0x7f50040b7d40 2024-03-16T20:45:46.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.127+0000 7f50027fc640 1 -- 172.21.15.122:0/2021538508 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff40ac7a0 con 0x7f50040b7d40 2024-03-16T20:45:46.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.127+0000 7f50027fc640 1 -- 172.21.15.122:0/2021538508 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff40acb30 con 0x7f50040b7d40 2024-03-16T20:45:46.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.127+0000 7f50092ff640 1 -- 172.21.15.122:0/2021538508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50040b7d40 msgr2=0x7f50040b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:46.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.127+0000 7f50092ff640 1 --2- 172.21.15.122:0/2021538508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50040b7d40 0x7f50040b6bd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40097c0 tx=0x7f4ff401c3e0 comp rx=0 tx=0).stop 2024-03-16T20:45:46.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.127+0000 7f50092ff640 1 -- 172.21.15.122:0/2021538508 shutdown_connections 2024-03-16T20:45:46.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.127+0000 7f50092ff640 1 --2- 172.21.15.122:0/2021538508 >> v2:172.21.15.195:3300/0 conn(0x7f5004148480 0x7f500414f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:46.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.127+0000 7f50092ff640 1 --2- 172.21.15.122:0/2021538508 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50040b7d40 0x7f50040b6bd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:46.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.127+0000 7f50092ff640 1 -- 172.21.15.122:0/2021538508 >> 172.21.15.122:0/2021538508 conn(0x7f500406fc00 msgr2=0x7f5004070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:46.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.127+0000 7f50092ff640 1 -- 172.21.15.122:0/2021538508 shutdown_connections 2024-03-16T20:45:46.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.127+0000 7f50092ff640 1 -- 172.21.15.122:0/2021538508 wait complete. 2024-03-16T20:45:46.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.127+0000 7f50092ff640 1 Processor -- start 2024-03-16T20:45:46.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.127+0000 7f50092ff640 1 -- start start 2024-03-16T20:45:46.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50092ff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50040b7540 0x7f5004138380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:46.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50092ff640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50040b7d40 0x7f500413a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:46.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50092ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5004148480 0x7f500413c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:46.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50092ff640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5004139f20 con 0x7f5004148480 2024-03-16T20:45:46.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50092ff640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50041388c0 con 0x7f50040b7d40 2024-03-16T20:45:46.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50092ff640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500413aa50 con 0x7f50040b7540 2024-03-16T20:45:46.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50037fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5004148480 0x7f500413c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:46.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50037fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5004148480 0x7f500413c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42394/0 (socket says 172.21.15.122:42394) 2024-03-16T20:45:46.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50037fe640 1 -- 172.21.15.122:0/334569610 learned_addr learned my addr 172.21.15.122:0/334569610 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:46.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f4ffa7fc640 1 --2- 172.21.15.122:0/334569610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50040b7d40 0x7f500413a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:46.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f5002ffd640 1 --2- 172.21.15.122:0/334569610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50040b7540 0x7f5004138380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:46.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50037fe640 1 -- 172.21.15.122:0/334569610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50040b7540 msgr2=0x7f5004138380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:46.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50037fe640 1 --2- 172.21.15.122:0/334569610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50040b7540 0x7f5004138380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:46.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50037fe640 1 -- 172.21.15.122:0/334569610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50040b7d40 msgr2=0x7f500413a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:46.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50037fe640 1 --2- 172.21.15.122:0/334569610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50040b7d40 0x7f500413a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:46.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50037fe640 1 -- 172.21.15.122:0/334569610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ffc009a70 con 0x7f5004148480 2024-03-16T20:45:46.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f4ffa7fc640 1 --2- 172.21.15.122:0/334569610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50040b7d40 0x7f500413a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:46.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50037fe640 1 --2- 172.21.15.122:0/334569610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5004148480 0x7f500413c9f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00efc0 tx=0x7f4ffc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:46.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f4ffbfff640 1 -- 172.21.15.122:0/334569610 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc009cd0 con 0x7f5004148480 2024-03-16T20:45:46.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f4ffbfff640 1 -- 172.21.15.122:0/334569610 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ffc00f040 con 0x7f5004148480 2024-03-16T20:45:46.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50092ff640 1 -- 172.21.15.122:0/334569610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff4009420 con 0x7f5004148480 2024-03-16T20:45:46.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f4ffbfff640 1 -- 172.21.15.122:0/334569610 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ffc0039a0 con 0x7f5004148480 2024-03-16T20:45:46.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50092ff640 1 -- 172.21.15.122:0/334569610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5004139c40 con 0x7f5004148480 2024-03-16T20:45:46.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f50092ff640 1 -- 172.21.15.122:0/334569610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50040b2db0 con 0x7f5004148480 2024-03-16T20:45:46.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f4ffbfff640 1 -- 172.21.15.122:0/334569610 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f4ffc014480 con 0x7f5004148480 2024-03-16T20:45:46.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f4ffbfff640 1 --2- 172.21.15.122:0/334569610 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4fe8066e40 0x7f4fe8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:46.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.131+0000 7f4ffbfff640 1 -- 172.21.15.122:0/334569610 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ffc0855b0 con 0x7f5004148480 2024-03-16T20:45:46.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.135+0000 7f5002ffd640 1 --2- 172.21.15.122:0/334569610 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4fe8066e40 0x7f4fe8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:46.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.135+0000 7f5002ffd640 1 --2- 172.21.15.122:0/334569610 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4fe8066e40 0x7f4fe8069300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4fec000f70 tx=0x7f4fec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:46.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.139+0000 7f4ffbfff640 1 -- 172.21.15.122:0/334569610 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4ffc052430 con 0x7f5004148480 2024-03-16T20:45:46.326 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.323+0000 7f50092ff640 1 -- 172.21.15.122:0/334569610 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f50040bb040 con 0x7f5004148480 2024-03-16T20:45:46.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.991+0000 7f4ffbfff640 1 -- 172.21.15.122:0/334569610 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v13) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4ffc055af0 con 0x7f5004148480 2024-03-16T20:45:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.995+0000 7f50092ff640 1 -- 172.21.15.122:0/334569610 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4fe8066e40 msgr2=0x7f4fe8069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.995+0000 7f50092ff640 1 --2- 172.21.15.122:0/334569610 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4fe8066e40 0x7f4fe8069300 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4fec000f70 tx=0x7f4fec009040 comp rx=0 tx=0).stop 2024-03-16T20:45:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.995+0000 7f50092ff640 1 -- 172.21.15.122:0/334569610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5004148480 msgr2=0x7f500413c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.995+0000 7f50092ff640 1 --2- 172.21.15.122:0/334569610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5004148480 0x7f500413c9f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4ffc00efc0 tx=0x7f4ffc00c490 comp rx=0 tx=0).stop 2024-03-16T20:45:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.995+0000 7f50092ff640 1 -- 172.21.15.122:0/334569610 shutdown_connections 2024-03-16T20:45:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.995+0000 7f50092ff640 1 --2- 172.21.15.122:0/334569610 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4fe8066e40 0x7f4fe8069300 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.995+0000 7f50092ff640 1 --2- 172.21.15.122:0/334569610 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5004148480 0x7f500413c9f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.995+0000 7f50092ff640 1 --2- 172.21.15.122:0/334569610 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f50040b7d40 0x7f500413a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.995+0000 7f50092ff640 1 --2- 172.21.15.122:0/334569610 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f50040b7540 0x7f5004138380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.995+0000 7f50092ff640 1 -- 172.21.15.122:0/334569610 >> 172.21.15.122:0/334569610 conn(0x7f500406fc00 msgr2=0x7f50040b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.995+0000 7f50092ff640 1 -- 172.21.15.122:0/334569610 shutdown_connections 2024-03-16T20:45:47.001 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:46.995+0000 7f50092ff640 1 -- 172.21.15.122:0/334569610 wait complete. 2024-03-16T20:45:47.018 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-03-16T20:45:47.018 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T20:45:47.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.131+0000 7ff48fa47640 1 Processor -- start 2024-03-16T20:45:47.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.131+0000 7ff48fa47640 1 -- start start 2024-03-16T20:45:47.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.131+0000 7ff48fa47640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff48806edd0 0x7ff488071b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.131+0000 7ff48fa47640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff48813f520 0x7ff48814f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.131+0000 7ff48fa47640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4880716b0 con 0x7ff488148500 2024-03-16T20:45:47.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.131+0000 7ff48fa47640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff488072570 con 0x7ff48813f520 2024-03-16T20:45:47.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.131+0000 7ff48fa47640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4880b36d0 con 0x7ff48806edd0 2024-03-16T20:45:47.136 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.131+0000 7ff48dfe4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff48813f520 0x7ff48814f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:47.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.131+0000 7ff48dfe4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7ff48813f520 0x7ff48814f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42406/0 (socket says 172.21.15.122:42406) 2024-03-16T20:45:47.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.131+0000 7ff48dfe4640 1 -- 172.21.15.122:0/1331159278 learned_addr learned my addr 172.21.15.122:0/1331159278 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:47.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.131+0000 7ff48d7e3640 1 --2- 172.21.15.122:0/1331159278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff48806edd0 0x7ff488071b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:47.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.131+0000 7ff48dfe4640 1 -- 172.21.15.122:0/1331159278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff48806edd0 msgr2=0x7ff488071b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:47.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.131+0000 7ff48dfe4640 1 --2- 172.21.15.122:0/1331159278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff48806edd0 0x7ff488071b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.131+0000 7ff48dfe4640 1 -- 172.21.15.122:0/1331159278 >> v1:172.21.15.195:6789/0 conn(0x7ff488148500 legacy=0x7ff48813ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:47.137 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.131+0000 7ff48dfe4640 1 -- 172.21.15.122:0/1331159278 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48806f1b0 con 0x7ff48813f520 2024-03-16T20:45:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48dfe4640 1 --2- 172.21.15.122:0/1331159278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff48813f520 0x7ff48814f880 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff480009e50 tx=0x7ff48001ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ac919ee06a3df8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff47ffff640 1 -- 172.21.15.122:0/1331159278 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4800a5920 con 0x7ff48813f520 2024-03-16T20:45:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff47ffff640 1 -- 172.21.15.122:0/1331159278 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4800a5a80 con 0x7ff48813f520 2024-03-16T20:45:47.138 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff47ffff640 1 -- 172.21.15.122:0/1331159278 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4800a5e10 con 0x7ff48813f520 2024-03-16T20:45:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 -- 172.21.15.122:0/1331159278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff48813f520 msgr2=0x7ff48814f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 --2- 172.21.15.122:0/1331159278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff48813f520 0x7ff48814f880 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff480009e50 tx=0x7ff48001ca30 comp rx=0 tx=0).stop 2024-03-16T20:45:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 -- 172.21.15.122:0/1331159278 shutdown_connections 2024-03-16T20:45:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 --2- 172.21.15.122:0/1331159278 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff48813f520 0x7ff48814f880 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 --2- 172.21.15.122:0/1331159278 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff48806edd0 0x7ff488071b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 -- 172.21.15.122:0/1331159278 >> 172.21.15.122:0/1331159278 conn(0x7ff48806c0a0 msgr2=0x7ff48806c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 -- 172.21.15.122:0/1331159278 shutdown_connections 2024-03-16T20:45:47.139 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 -- 172.21.15.122:0/1331159278 wait complete. 2024-03-16T20:45:47.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 Processor -- start 2024-03-16T20:45:47.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 -- start start 2024-03-16T20:45:47.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff48806edd0 0x7ff4880b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff48813f520 0x7ff488135180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff488148500 0x7ff488130750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4881de6e0 con 0x7ff48806edd0 2024-03-16T20:45:47.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4881ddc00 con 0x7ff488148500 2024-03-16T20:45:47.142 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4881ddee0 con 0x7ff48813f520 2024-03-16T20:45:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48d7e3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff48806edd0 0x7ff4880b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48d7e3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff48806edd0 0x7ff4880b8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42416/0 (socket says 172.21.15.122:42416) 2024-03-16T20:45:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48d7e3640 1 -- 172.21.15.122:0/2986059885 learned_addr learned my addr 172.21.15.122:0/2986059885 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48dfe4640 1 --2- 172.21.15.122:0/2986059885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff488148500 0x7ff488130750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48cfe2640 1 --2- 172.21.15.122:0/2986059885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff48813f520 0x7ff488135180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48d7e3640 1 -- 172.21.15.122:0/2986059885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff48813f520 msgr2=0x7ff488135180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48d7e3640 1 --2- 172.21.15.122:0/2986059885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff48813f520 0x7ff488135180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48d7e3640 1 -- 172.21.15.122:0/2986059885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff488148500 msgr2=0x7ff488130750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48d7e3640 1 --2- 172.21.15.122:0/2986059885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff488148500 0x7ff488130750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48d7e3640 1 -- 172.21.15.122:0/2986059885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff480009b30 con 0x7ff48806edd0 2024-03-16T20:45:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48cfe2640 1 --2- 172.21.15.122:0/2986059885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff48813f520 0x7ff488135180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48d7e3640 1 --2- 172.21.15.122:0/2986059885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff48806edd0 0x7ff4880b8af0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff48400d5f0 tx=0x7ff48400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:47.143 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff47dffb640 1 -- 172.21.15.122:0/2986059885 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4840125f0 con 0x7ff48806edd0 2024-03-16T20:45:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff47dffb640 1 -- 172.21.15.122:0/2986059885 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff484012750 con 0x7ff48806edd0 2024-03-16T20:45:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 -- 172.21.15.122:0/2986059885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4881de1c0 con 0x7ff48806edd0 2024-03-16T20:45:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff47dffb640 1 -- 172.21.15.122:0/2986059885 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484012b00 con 0x7ff48806edd0 2024-03-16T20:45:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.135+0000 7ff48fa47640 1 -- 172.21.15.122:0/2986059885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4881dd9a0 con 0x7ff48806edd0 2024-03-16T20:45:47.144 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.139+0000 7ff48fa47640 1 -- 172.21.15.122:0/2986059885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4881de3e0 con 0x7ff48806edd0 2024-03-16T20:45:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.139+0000 7ff47dffb640 1 -- 172.21.15.122:0/2986059885 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7ff484012c60 con 0x7ff48806edd0 2024-03-16T20:45:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.139+0000 7ff47dffb640 1 --2- 172.21.15.122:0/2986059885 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff46c066e40 0x7ff46c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.139+0000 7ff47dffb640 1 -- 172.21.15.122:0/2986059885 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff484085960 con 0x7ff48806edd0 2024-03-16T20:45:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.139+0000 7ff48cfe2640 1 --2- 172.21.15.122:0/2986059885 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff46c066e40 0x7ff46c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:47.145 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.139+0000 7ff48cfe2640 1 --2- 172.21.15.122:0/2986059885 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff46c066e40 0x7ff46c069300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff4880b92c0 tx=0x7ff478009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:47.147 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.143+0000 7ff47dffb640 1 -- 172.21.15.122:0/2986059885 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff4840528c0 con 0x7ff48806edd0 2024-03-16T20:45:47.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.323+0000 7ff48fa47640 1 -- 172.21.15.122:0/2986059885 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4880b5f80 con 0x7ff48806edd0 2024-03-16T20:45:47.329 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.323+0000 7ff47dffb640 1 -- 172.21.15.122:0/2986059885 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+3401 (secure 0 0 0) 0x7ff484055f80 con 0x7ff48806edd0 2024-03-16T20:45:47.330 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:47.330 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3638578490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3638578490},{"type":"v1","addr":"172.21.15.195:6835","nonce":3638578490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T20:45:44.280433+0000","modified":"2024-03-16T20:45:46.325382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4995":{"gid":4995,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6833/1841055069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1841055069},{"type":"v1","addr":"172.21.15.122:6833","nonce":1841055069}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4995,"qdb_cluster":[4995]},"id":2}]} 2024-03-16T20:45:47.330 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 13 2024-03-16T20:45:47.331 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.327+0000 7ff48fa47640 1 -- 172.21.15.122:0/2986059885 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff46c066e40 msgr2=0x7ff46c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:47.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.327+0000 7ff48fa47640 1 --2- 172.21.15.122:0/2986059885 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff46c066e40 0x7ff46c069300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff4880b92c0 tx=0x7ff478009040 comp rx=0 tx=0).stop 2024-03-16T20:45:47.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.327+0000 7ff48fa47640 1 -- 172.21.15.122:0/2986059885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff48806edd0 msgr2=0x7ff4880b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:47.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.327+0000 7ff48fa47640 1 --2- 172.21.15.122:0/2986059885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff48806edd0 0x7ff4880b8af0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff48400d5f0 tx=0x7ff48400dab0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.327+0000 7ff48fa47640 1 -- 172.21.15.122:0/2986059885 shutdown_connections 2024-03-16T20:45:47.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.327+0000 7ff48fa47640 1 --2- 172.21.15.122:0/2986059885 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff46c066e40 0x7ff46c069300 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.327+0000 7ff48fa47640 1 --2- 172.21.15.122:0/2986059885 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff488148500 0x7ff488130750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.327+0000 7ff48fa47640 1 --2- 172.21.15.122:0/2986059885 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff48813f520 0x7ff488135180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.327+0000 7ff48fa47640 1 --2- 172.21.15.122:0/2986059885 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff48806edd0 0x7ff4880b8af0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.327+0000 7ff48fa47640 1 -- 172.21.15.122:0/2986059885 >> 172.21.15.122:0/2986059885 conn(0x7ff48806c0a0 msgr2=0x7ff4881488e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:47.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.327+0000 7ff48fa47640 1 -- 172.21.15.122:0/2986059885 shutdown_connections 2024-03-16T20:45:47.332 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.327+0000 7ff48fa47640 1 -- 172.21.15.122:0/2986059885 wait complete. 2024-03-16T20:45:47.346 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:47.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.459+0000 7efe2018f640 1 Processor -- start 2024-03-16T20:45:47.462 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.459+0000 7efe2018f640 1 -- start start 2024-03-16T20:45:47.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.459+0000 7efe2018f640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7efe1814e7b0 0x7efe18150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.459+0000 7efe2018f640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe180b0ff0 con 0x7efe180afdc0 2024-03-16T20:45:47.463 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.459+0000 7efe2018f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe180b0220 con 0x7efe181486e0 2024-03-16T20:45:47.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.459+0000 7efe2018f640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe1813f700 con 0x7efe1814e7b0 2024-03-16T20:45:47.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.459+0000 7efe1e72c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7efe1814e7b0 0x7efe18150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:47.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.459+0000 7efe1e72c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7efe1814e7b0 0x7efe18150ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42420/0 (socket says 172.21.15.122:42420) 2024-03-16T20:45:47.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.459+0000 7efe1e72c640 1 -- 172.21.15.122:0/4161030466 learned_addr learned my addr 172.21.15.122:0/4161030466 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:47.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.459+0000 7efe1e72c640 1 -- 172.21.15.122:0/4161030466 >> v1:172.21.15.195:6789/0 conn(0x7efe181486e0 legacy=0x7efe1813eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:47.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.459+0000 7efe1e72c640 1 -- 172.21.15.122:0/4161030466 >> v1:172.21.15.122:6789/0 conn(0x7efe180afdc0 legacy=0x7efe18148ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:47.464 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.459+0000 7efe1e72c640 1 -- 172.21.15.122:0/4161030466 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe180b1ce0 con 0x7efe1814e7b0 2024-03-16T20:45:47.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.459+0000 7efe1e72c640 1 --2- 172.21.15.122:0/4161030466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe1814e7b0 0x7efe18150ba0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efe10009e50 tx=0x7efe1001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5180a14861e1df6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:47.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.459+0000 7efe1cf29640 1 -- 172.21.15.122:0/4161030466 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe100a5730 con 0x7efe1814e7b0 2024-03-16T20:45:47.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.459+0000 7efe1cf29640 1 -- 172.21.15.122:0/4161030466 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe100a5890 con 0x7efe1814e7b0 2024-03-16T20:45:47.466 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.459+0000 7efe1cf29640 1 -- 172.21.15.122:0/4161030466 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe100a5c20 con 0x7efe1814e7b0 2024-03-16T20:45:47.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe2018f640 1 -- 172.21.15.122:0/4161030466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe1814e7b0 msgr2=0x7efe18150ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:47.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe2018f640 1 --2- 172.21.15.122:0/4161030466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe1814e7b0 0x7efe18150ba0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efe10009e50 tx=0x7efe1001c9a0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe2018f640 1 -- 172.21.15.122:0/4161030466 shutdown_connections 2024-03-16T20:45:47.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe2018f640 1 --2- 172.21.15.122:0/4161030466 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe1814e7b0 0x7efe18150ba0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe2018f640 1 -- 172.21.15.122:0/4161030466 >> 172.21.15.122:0/4161030466 conn(0x7efe1806fed0 msgr2=0x7efe180702e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:47.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe2018f640 1 -- 172.21.15.122:0/4161030466 shutdown_connections 2024-03-16T20:45:47.467 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe2018f640 1 -- 172.21.15.122:0/4161030466 wait complete. 2024-03-16T20:45:47.469 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe2018f640 1 Processor -- start 2024-03-16T20:45:47.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe2018f640 1 -- start start 2024-03-16T20:45:47.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe2018f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe180afdc0 0x7efe181e0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe2018f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe181486e0 0x7efe181393f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe2018f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe1814e7b0 0x7efe18140ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe2018f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1813fe00 con 0x7efe180afdc0 2024-03-16T20:45:47.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe2018f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1813fbb0 con 0x7efe181486e0 2024-03-16T20:45:47.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe2018f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe18141f70 con 0x7efe1814e7b0 2024-03-16T20:45:47.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe1df2b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe180afdc0 0x7efe181e0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:47.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe1df2b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe180afdc0 0x7efe181e0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42436/0 (socket says 172.21.15.122:42436) 2024-03-16T20:45:47.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe1df2b640 1 -- 172.21.15.122:0/3990057665 learned_addr learned my addr 172.21.15.122:0/3990057665 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:47.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe1d72a640 1 --2- 172.21.15.122:0/3990057665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe181486e0 0x7efe181393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:47.470 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe1df2b640 1 -- 172.21.15.122:0/3990057665 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe1814e7b0 msgr2=0x7efe18140ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:47.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe1df2b640 1 --2- 172.21.15.122:0/3990057665 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe1814e7b0 0x7efe18140ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe1df2b640 1 -- 172.21.15.122:0/3990057665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe181486e0 msgr2=0x7efe181393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:47.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe1df2b640 1 --2- 172.21.15.122:0/3990057665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe181486e0 0x7efe181393f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe1df2b640 1 -- 172.21.15.122:0/3990057665 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe10009b30 con 0x7efe180afdc0 2024-03-16T20:45:47.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe1df2b640 1 --2- 172.21.15.122:0/3990057665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe180afdc0 0x7efe181e0e50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efe1400c450 tx=0x7efe1400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:47.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe0e7fc640 1 -- 172.21.15.122:0/3990057665 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe140052d0 con 0x7efe180afdc0 2024-03-16T20:45:47.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe0e7fc640 1 -- 172.21.15.122:0/3990057665 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe14005430 con 0x7efe180afdc0 2024-03-16T20:45:47.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe2018f640 1 -- 172.21.15.122:0/3990057665 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe18141220 con 0x7efe180afdc0 2024-03-16T20:45:47.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe0e7fc640 1 -- 172.21.15.122:0/3990057665 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14007260 con 0x7efe180afdc0 2024-03-16T20:45:47.471 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.463+0000 7efe2018f640 1 -- 172.21.15.122:0/3990057665 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe18145f90 con 0x7efe180afdc0 2024-03-16T20:45:47.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.467+0000 7efe2018f640 1 -- 172.21.15.122:0/3990057665 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe18145b10 con 0x7efe180afdc0 2024-03-16T20:45:47.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.467+0000 7efe0e7fc640 1 -- 172.21.15.122:0/3990057665 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7efe140073c0 con 0x7efe180afdc0 2024-03-16T20:45:47.472 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.467+0000 7efe0e7fc640 1 --2- 172.21.15.122:0/3990057665 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efdfc066e40 0x7efdfc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.467+0000 7efe0e7fc640 1 -- 172.21.15.122:0/3990057665 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe140842a0 con 0x7efe180afdc0 2024-03-16T20:45:47.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.467+0000 7efe1d72a640 1 --2- 172.21.15.122:0/3990057665 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efdfc066e40 0x7efdfc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:47.473 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.467+0000 7efe1d72a640 1 --2- 172.21.15.122:0/3990057665 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efdfc066e40 0x7efdfc069300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efe18141b50 tx=0x7efe08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:47.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.471+0000 7efe0e7fc640 1 -- 172.21.15.122:0/3990057665 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7efe18145b10 con 0x7efe180afdc0 2024-03-16T20:45:47.636 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.631+0000 7efe2018f640 1 -- 172.21.15.122:0/3990057665 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe18145b10 con 0x7efe180afdc0 2024-03-16T20:45:47.639 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.635+0000 7efe0e7fc640 1 -- 172.21.15.122:0/3990057665 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+16705 (secure 0 0 0) 0x7efe18145b10 con 0x7efe180afdc0 2024-03-16T20:45:47.639 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:47.639 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:45.942613+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:42.678541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:43.274469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:47.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.635+0000 7efe2018f640 1 -- 172.21.15.122:0/3990057665 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efdfc066e40 msgr2=0x7efdfc069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:47.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.635+0000 7efe2018f640 1 --2- 172.21.15.122:0/3990057665 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efdfc066e40 0x7efdfc069300 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efe18141b50 tx=0x7efe08009040 comp rx=0 tx=0).stop 2024-03-16T20:45:47.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.635+0000 7efe2018f640 1 -- 172.21.15.122:0/3990057665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe180afdc0 msgr2=0x7efe181e0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:47.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.635+0000 7efe2018f640 1 --2- 172.21.15.122:0/3990057665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe180afdc0 0x7efe181e0e50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7efe1400c450 tx=0x7efe1400c910 comp rx=0 tx=0).stop 2024-03-16T20:45:47.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.635+0000 7efe2018f640 1 -- 172.21.15.122:0/3990057665 shutdown_connections 2024-03-16T20:45:47.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.635+0000 7efe2018f640 1 --2- 172.21.15.122:0/3990057665 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7efdfc066e40 0x7efdfc069300 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.635+0000 7efe2018f640 1 --2- 172.21.15.122:0/3990057665 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7efe1814e7b0 0x7efe18140ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.641 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.635+0000 7efe2018f640 1 --2- 172.21.15.122:0/3990057665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7efe181486e0 0x7efe181393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.635+0000 7efe2018f640 1 --2- 172.21.15.122:0/3990057665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe180afdc0 0x7efe181e0e50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.635+0000 7efe2018f640 1 -- 172.21.15.122:0/3990057665 >> 172.21.15.122:0/3990057665 conn(0x7efe1806fed0 msgr2=0x7efe180b8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:47.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.635+0000 7efe2018f640 1 -- 172.21.15.122:0/3990057665 shutdown_connections 2024-03-16T20:45:47.642 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.635+0000 7efe2018f640 1 -- 172.21.15.122:0/3990057665 wait complete. 2024-03-16T20:45:47.663 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-16T20:45:47.663 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T20:45:47.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.771+0000 7f191a21a640 1 Processor -- start 2024-03-16T20:45:47.777 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.771+0000 7f191a21a640 1 -- start start 2024-03-16T20:45:47.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f191a21a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1914066fa0 0x7f191409a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f191a21a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1914072df0 0x7f1914144f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f191a21a640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f19140b5060 0x7f191413ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f191a21a640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19141454b0 con 0x7f19140b5060 2024-03-16T20:45:47.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f191a21a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1914099d90 con 0x7f1914072df0 2024-03-16T20:45:47.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f191a21a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191409bc00 con 0x7f1914066fa0 2024-03-16T20:45:47.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f1913fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f19140b5060 0x7f191413ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:47.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f1913fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f19140b5060 0x7f191413ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42440/0 (socket says 172.21.15.122:42440) 2024-03-16T20:45:47.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f1913fff640 1 -- 172.21.15.122:0/1248546017 learned_addr learned my addr 172.21.15.122:0/1248546017 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:47.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f1912ffd640 1 --2- 172.21.15.122:0/1248546017 >> v2:172.21.15.195:3300/0 conn(0x7f1914072df0 0x7f1914144f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:47.780 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f19137fe640 1 --2- 172.21.15.122:0/1248546017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1914066fa0 0x7f191409a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:47.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f1913fff640 1 -- 172.21.15.122:0/1248546017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1914066fa0 msgr2=0x7f191409a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:47.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f1913fff640 1 --2- 172.21.15.122:0/1248546017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1914066fa0 0x7f191409a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f1913fff640 1 -- 172.21.15.122:0/1248546017 >> v2:172.21.15.195:3300/0 conn(0x7f1914072df0 msgr2=0x7f1914144f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:47.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f1913fff640 1 --2- 172.21.15.122:0/1248546017 >> v2:172.21.15.195:3300/0 conn(0x7f1914072df0 0x7f1914144f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.781 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f1913fff640 1 -- 172.21.15.122:0/1248546017 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1914067380 con 0x7f19140b5060 2024-03-16T20:45:47.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f1913fff640 1 --2- 172.21.15.122:0/1248546017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19140b5060 0x7f191413ee30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f19140b0350 tx=0x7f190c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d73089660a5dedd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:47.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f19127fc640 1 -- 172.21.15.122:0/1248546017 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c0a56f0 con 0x7f19140b5060 2024-03-16T20:45:47.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f19127fc640 1 -- 172.21.15.122:0/1248546017 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190c0a5850 con 0x7f19140b5060 2024-03-16T20:45:47.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f19127fc640 1 -- 172.21.15.122:0/1248546017 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c0a5be0 con 0x7f19140b5060 2024-03-16T20:45:47.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f191a21a640 1 -- 172.21.15.122:0/1248546017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19140b5060 msgr2=0x7f191413ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:47.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f191a21a640 1 --2- 172.21.15.122:0/1248546017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19140b5060 0x7f191413ee30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f19140b0350 tx=0x7f190c01c960 comp rx=0 tx=0).stop 2024-03-16T20:45:47.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f191a21a640 1 -- 172.21.15.122:0/1248546017 shutdown_connections 2024-03-16T20:45:47.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f191a21a640 1 --2- 172.21.15.122:0/1248546017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f19140b5060 0x7f191413ee30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f191a21a640 1 --2- 172.21.15.122:0/1248546017 >> v2:172.21.15.195:3300/0 conn(0x7f1914072df0 0x7f1914144f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f191a21a640 1 --2- 172.21.15.122:0/1248546017 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1914066fa0 0x7f191409a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f191a21a640 1 -- 172.21.15.122:0/1248546017 >> 172.21.15.122:0/1248546017 conn(0x7f191406d320 msgr2=0x7f191406d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:47.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f191a21a640 1 -- 172.21.15.122:0/1248546017 shutdown_connections 2024-03-16T20:45:47.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.775+0000 7f191a21a640 1 -- 172.21.15.122:0/1248546017 wait complete. 2024-03-16T20:45:47.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f191a21a640 1 Processor -- start 2024-03-16T20:45:47.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f191a21a640 1 -- start start 2024-03-16T20:45:47.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f191a21a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1914066fa0 0x7f19140b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f191a21a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1914072df0 0x7f19140b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f191a21a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19140b5060 0x7f19141dde80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f191a21a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1914142c20 con 0x7f1914066fa0 2024-03-16T20:45:47.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f191a21a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19141dd9f0 con 0x7f19140b5060 2024-03-16T20:45:47.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f191a21a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191413fab0 con 0x7f1914072df0 2024-03-16T20:45:47.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f19137fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1914066fa0 0x7f19140b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:47.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f19137fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1914066fa0 0x7f19140b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42456/0 (socket says 172.21.15.122:42456) 2024-03-16T20:45:47.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f19137fe640 1 -- 172.21.15.122:0/1718665416 learned_addr learned my addr 172.21.15.122:0/1718665416 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:47.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f19137fe640 1 -- 172.21.15.122:0/1718665416 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1914072df0 msgr2=0x7f19140b2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:47.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f19137fe640 1 --2- 172.21.15.122:0/1718665416 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1914072df0 0x7f19140b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f1913fff640 1 --2- 172.21.15.122:0/1718665416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19140b5060 0x7f19141dde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:47.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f19137fe640 1 -- 172.21.15.122:0/1718665416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19140b5060 msgr2=0x7f19141dde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:47.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f19137fe640 1 --2- 172.21.15.122:0/1718665416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19140b5060 0x7f19141dde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:47.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f19137fe640 1 -- 172.21.15.122:0/1718665416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c009b30 con 0x7f1914066fa0 2024-03-16T20:45:47.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f1913fff640 1 --2- 172.21.15.122:0/1718665416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19140b5060 0x7f19141dde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:47.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f19137fe640 1 --2- 172.21.15.122:0/1718665416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1914066fa0 0x7f19140b22e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00d5f0 tx=0x7f18fc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:47.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f190bfff640 1 -- 172.21.15.122:0/1718665416 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc0125f0 con 0x7f1914066fa0 2024-03-16T20:45:47.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f190bfff640 1 -- 172.21.15.122:0/1718665416 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc012750 con 0x7f1914066fa0 2024-03-16T20:45:47.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f191a21a640 1 -- 172.21.15.122:0/1718665416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f191409f380 con 0x7f1914066fa0 2024-03-16T20:45:47.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f190bfff640 1 -- 172.21.15.122:0/1718665416 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc012ae0 con 0x7f1914066fa0 2024-03-16T20:45:47.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f191a21a640 1 -- 172.21.15.122:0/1718665416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19140b3370 con 0x7f1914066fa0 2024-03-16T20:45:47.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.779+0000 7f191a21a640 1 -- 172.21.15.122:0/1718665416 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1914075280 con 0x7f1914066fa0 2024-03-16T20:45:47.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.783+0000 7f190bfff640 1 -- 172.21.15.122:0/1718665416 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f18fc012c40 con 0x7f1914066fa0 2024-03-16T20:45:47.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.783+0000 7f190bfff640 1 --2- 172.21.15.122:0/1718665416 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f18f4066e40 0x7f18f4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:47.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.783+0000 7f190bfff640 1 -- 172.21.15.122:0/1718665416 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18fc084f50 con 0x7f1914066fa0 2024-03-16T20:45:47.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.783+0000 7f1912ffd640 1 --2- 172.21.15.122:0/1718665416 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f18f4066e40 0x7f18f4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:47.788 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.783+0000 7f1912ffd640 1 --2- 172.21.15.122:0/1718665416 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f18f4066e40 0x7f18f4069300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f191413f990 tx=0x7f1904008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:47.791 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:47.787+0000 7f190bfff640 1 -- 172.21.15.122:0/1718665416 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f18fc051d30 con 0x7f1914066fa0 2024-03-16T20:45:48.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.043+0000 7f191a21a640 1 -- 172.21.15.122:0/1718665416 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f191414f6a0 con 0x7f18f4066e40 2024-03-16T20:45:48.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.059+0000 7f190bfff640 1 -- 172.21.15.122:0/1718665416 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237200 (secure 0 0 0) 0x7f191414f6a0 con 0x7f18f4066e40 2024-03-16T20:45:48.068 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:48.070 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-03-16T20:45:48.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.063+0000 7f191a21a640 1 -- 172.21.15.122:0/1718665416 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f18f4066e40 msgr2=0x7f18f4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.063+0000 7f191a21a640 1 --2- 172.21.15.122:0/1718665416 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f18f4066e40 0x7f18f4069300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f191413f990 tx=0x7f1904008040 comp rx=0 tx=0).stop 2024-03-16T20:45:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.063+0000 7f191a21a640 1 -- 172.21.15.122:0/1718665416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1914066fa0 msgr2=0x7f19140b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.063+0000 7f191a21a640 1 --2- 172.21.15.122:0/1718665416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1914066fa0 0x7f19140b22e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00d5f0 tx=0x7f18fc00dab0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.063+0000 7f191a21a640 1 -- 172.21.15.122:0/1718665416 shutdown_connections 2024-03-16T20:45:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.063+0000 7f191a21a640 1 --2- 172.21.15.122:0/1718665416 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f18f4066e40 0x7f18f4069300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.063+0000 7f191a21a640 1 --2- 172.21.15.122:0/1718665416 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f19140b5060 0x7f19141dde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.063+0000 7f191a21a640 1 --2- 172.21.15.122:0/1718665416 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1914072df0 0x7f19140b2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.063+0000 7f191a21a640 1 --2- 172.21.15.122:0/1718665416 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1914066fa0 0x7f19140b22e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.063+0000 7f191a21a640 1 -- 172.21.15.122:0/1718665416 >> 172.21.15.122:0/1718665416 conn(0x7f191406d320 msgr2=0x7f19140b7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.063+0000 7f191a21a640 1 -- 172.21.15.122:0/1718665416 shutdown_connections 2024-03-16T20:45:48.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.063+0000 7f191a21a640 1 -- 172.21.15.122:0/1718665416 wait complete. 2024-03-16T20:45:48.082 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2024-03-16T20:45:46.590349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236248,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738472,"statfs":{"total":773094113280,"available":772852195328,"internally_reserved":0,"allocated":3481600,"data_stored":2008928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001173"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":179,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-16T20:45:39.263313+0000","last_change":"2024-03-16T20:44:54.293418+0000","last_active":"2024-03-16T20:45:39.263313+0000","last_peered":"2024-03-16T20:45:39.263313+0000","last_clean":"2024-03-16T20:45:39.263313+0000","last_became_active":"2024-03-16T20:44:54.291347+0000","last_became_peered":"2024-03-16T20:44:54.291347+0000","last_unstale":"2024-03-16T20:45:39.263313+0000","last_undegraded":"2024-03-16T20:45:39.263313+0000","last_fullsized":"2024-03-16T20:45:39.263313+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_clean_scrub_stamp":"2024-03-16T20:44:53.077639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:29:18.318494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":693892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31468,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":286720,"data_stored":103524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31468,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":286720,"data_stored":103524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31468,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":286720,"data_stored":103524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":103524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":286720,"data_stored":103524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":103524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":693892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T20:45:48.086 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T20:45:48.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.227+0000 7fc657256640 1 Processor -- start 2024-03-16T20:45:48.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.227+0000 7fc657256640 1 -- start start 2024-03-16T20:45:48.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc657256640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc6500b4c50 0x7fc6500b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:48.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc657256640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc650140e10 0x7fc650141290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:48.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc657256640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc650144160 0x7fc6501445f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:48.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc657256640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6500b1370 con 0x7fc6500b4c50 2024-03-16T20:45:48.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc657256640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc650144b30 con 0x7fc650140e10 2024-03-16T20:45:48.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc657256640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc650143ca0 con 0x7fc650144160 2024-03-16T20:45:48.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc654ff2640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc6500b4c50 0x7fc6500b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:48.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc654ff2640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc6500b4c50 0x7fc6500b5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42460/0 (socket says 172.21.15.122:42460) 2024-03-16T20:45:48.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc654ff2640 1 -- 172.21.15.122:0/2408770809 learned_addr learned my addr 172.21.15.122:0/2408770809 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:48.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc64ffff640 1 --2- 172.21.15.122:0/2408770809 >> v2:172.21.15.195:3300/0 conn(0x7fc650140e10 0x7fc650141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:48.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc6557f3640 1 --2- 172.21.15.122:0/2408770809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc650144160 0x7fc6501445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:48.236 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc654ff2640 1 -- 172.21.15.122:0/2408770809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc650144160 msgr2=0x7fc6501445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:48.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc654ff2640 1 --2- 172.21.15.122:0/2408770809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc650144160 0x7fc6501445f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc654ff2640 1 -- 172.21.15.122:0/2408770809 >> v2:172.21.15.195:3300/0 conn(0x7fc650140e10 msgr2=0x7fc650141290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:48.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc654ff2640 1 --2- 172.21.15.122:0/2408770809 >> v2:172.21.15.195:3300/0 conn(0x7fc650140e10 0x7fc650141290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc654ff2640 1 -- 172.21.15.122:0/2408770809 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6500b4340 con 0x7fc6500b4c50 2024-03-16T20:45:48.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc6557f3640 1 --2- 172.21.15.122:0/2408770809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc650144160 0x7fc6501445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:48.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc654ff2640 1 --2- 172.21.15.122:0/2408770809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6500b4c50 0x7fc6500b5030 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc650067570 tx=0x7fc64401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f8390ac8f394fe server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:48.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc64f7fe640 1 -- 172.21.15.122:0/2408770809 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440ab600 con 0x7fc6500b4c50 2024-03-16T20:45:48.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc64f7fe640 1 -- 172.21.15.122:0/2408770809 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6440a7050 con 0x7fc6500b4c50 2024-03-16T20:45:48.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.231+0000 7fc64f7fe640 1 -- 172.21.15.122:0/2408770809 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440ab980 con 0x7fc6500b4c50 2024-03-16T20:45:48.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 -- 172.21.15.122:0/2408770809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6500b4c50 msgr2=0x7fc6500b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:48.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 --2- 172.21.15.122:0/2408770809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6500b4c50 0x7fc6500b5030 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc650067570 tx=0x7fc64401c490 comp rx=0 tx=0).stop 2024-03-16T20:45:48.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 -- 172.21.15.122:0/2408770809 shutdown_connections 2024-03-16T20:45:48.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 --2- 172.21.15.122:0/2408770809 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc650144160 0x7fc6501445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 --2- 172.21.15.122:0/2408770809 >> v2:172.21.15.195:3300/0 conn(0x7fc650140e10 0x7fc650141290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 --2- 172.21.15.122:0/2408770809 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6500b4c50 0x7fc6500b5030 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 -- 172.21.15.122:0/2408770809 >> 172.21.15.122:0/2408770809 conn(0x7fc6500b62a0 msgr2=0x7fc6500b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:48.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 -- 172.21.15.122:0/2408770809 shutdown_connections 2024-03-16T20:45:48.239 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 -- 172.21.15.122:0/2408770809 wait complete. 2024-03-16T20:45:48.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 Processor -- start 2024-03-16T20:45:48.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 -- start start 2024-03-16T20:45:48.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6500b4c50 0x7fc6501469e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:48.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc650140e10 0x7fc6501e2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:48.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc650144160 0x7fc6501e3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:48.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6500b3640 con 0x7fc6500b4c50 2024-03-16T20:45:48.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6500b2d30 con 0x7fc650144160 2024-03-16T20:45:48.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6501387d0 con 0x7fc650140e10 2024-03-16T20:45:48.242 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc654ff2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6500b4c50 0x7fc6501469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc654ff2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6500b4c50 0x7fc6501469e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:42466/0 (socket says 172.21.15.122:42466) 2024-03-16T20:45:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc654ff2640 1 -- 172.21.15.122:0/4212546613 learned_addr learned my addr 172.21.15.122:0/4212546613 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc64ffff640 1 --2- 172.21.15.122:0/4212546613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc650140e10 0x7fc6501e2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc6557f3640 1 --2- 172.21.15.122:0/4212546613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc650144160 0x7fc6501e3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc654ff2640 1 -- 172.21.15.122:0/4212546613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc650140e10 msgr2=0x7fc6501e2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc654ff2640 1 --2- 172.21.15.122:0/4212546613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc650140e10 0x7fc6501e2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc654ff2640 1 -- 172.21.15.122:0/4212546613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc650144160 msgr2=0x7fc6501e3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc654ff2640 1 --2- 172.21.15.122:0/4212546613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc650144160 0x7fc6501e3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc654ff2640 1 -- 172.21.15.122:0/4212546613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6440094d0 con 0x7fc6500b4c50 2024-03-16T20:45:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc6557f3640 1 --2- 172.21.15.122:0/4212546613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc650144160 0x7fc6501e3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc654ff2640 1 --2- 172.21.15.122:0/4212546613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6500b4c50 0x7fc6501469e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc6440ae040 tx=0x7fc64401acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc64d7fa640 1 -- 172.21.15.122:0/4212546613 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440b6620 con 0x7fc6500b4c50 2024-03-16T20:45:48.243 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc64d7fa640 1 -- 172.21.15.122:0/4212546613 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6440abe90 con 0x7fc6500b4c50 2024-03-16T20:45:48.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 -- 172.21.15.122:0/4212546613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6500b2f80 con 0x7fc6500b4c50 2024-03-16T20:45:48.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc64d7fa640 1 -- 172.21.15.122:0/4212546613 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440b69a0 con 0x7fc6500b4c50 2024-03-16T20:45:48.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.235+0000 7fc657256640 1 -- 172.21.15.122:0/4212546613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6501384f0 con 0x7fc6500b4c50 2024-03-16T20:45:48.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.239+0000 7fc657256640 1 -- 172.21.15.122:0/4212546613 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6500b34a0 con 0x7fc6500b4c50 2024-03-16T20:45:48.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.239+0000 7fc64d7fa640 1 -- 172.21.15.122:0/4212546613 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fc6440b2040 con 0x7fc6500b4c50 2024-03-16T20:45:48.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.239+0000 7fc64d7fa640 1 --2- 172.21.15.122:0/4212546613 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc634066e40 0x7fc634069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:48.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.239+0000 7fc64d7fa640 1 -- 172.21.15.122:0/4212546613 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc64411eeb0 con 0x7fc6500b4c50 2024-03-16T20:45:48.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.239+0000 7fc64ffff640 1 --2- 172.21.15.122:0/4212546613 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc634066e40 0x7fc634069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:48.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.239+0000 7fc64ffff640 1 --2- 172.21.15.122:0/4212546613 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc634066e40 0x7fc634069300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00ca40 tx=0x7fc63c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:48.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.247+0000 7fc64d7fa640 1 -- 172.21.15.122:0/4212546613 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc6500b34a0 con 0x7fc6500b4c50 2024-03-16T20:45:48.406 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.399+0000 7fc657256640 1 -- 172.21.15.122:0/4212546613 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc6500b34a0 con 0x7fc634066e40 2024-03-16T20:45:48.421 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.415+0000 7fc64d7fa640 1 -- 172.21.15.122:0/4212546613 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237200 (secure 0 0 0) 0x7fc6500b34a0 con 0x7fc634066e40 2024-03-16T20:45:48.424 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:48.425 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-03-16T20:45:48.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.419+0000 7fc657256640 1 -- 172.21.15.122:0/4212546613 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc634066e40 msgr2=0x7fc634069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:48.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.419+0000 7fc657256640 1 --2- 172.21.15.122:0/4212546613 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc634066e40 0x7fc634069300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc63c00ca40 tx=0x7fc63c008040 comp rx=0 tx=0).stop 2024-03-16T20:45:48.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.419+0000 7fc657256640 1 -- 172.21.15.122:0/4212546613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6500b4c50 msgr2=0x7fc6501469e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:48.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.419+0000 7fc657256640 1 --2- 172.21.15.122:0/4212546613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6500b4c50 0x7fc6501469e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc6440ae040 tx=0x7fc64401acd0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.419+0000 7fc657256640 1 -- 172.21.15.122:0/4212546613 shutdown_connections 2024-03-16T20:45:48.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.419+0000 7fc657256640 1 --2- 172.21.15.122:0/4212546613 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc634066e40 0x7fc634069300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.419+0000 7fc657256640 1 --2- 172.21.15.122:0/4212546613 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc650144160 0x7fc6501e3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.419+0000 7fc657256640 1 --2- 172.21.15.122:0/4212546613 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc650140e10 0x7fc6501e2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.419+0000 7fc657256640 1 --2- 172.21.15.122:0/4212546613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc6500b4c50 0x7fc6501469e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.419+0000 7fc657256640 1 -- 172.21.15.122:0/4212546613 >> 172.21.15.122:0/4212546613 conn(0x7fc6500b62a0 msgr2=0x7fc65013e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:48.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.419+0000 7fc657256640 1 -- 172.21.15.122:0/4212546613 shutdown_connections 2024-03-16T20:45:48.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.419+0000 7fc657256640 1 -- 172.21.15.122:0/4212546613 wait complete. 2024-03-16T20:45:48.441 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":51,"stamp":"2024-03-16T20:45:46.590349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":41,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236248,"kb_used_data":3400,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738472,"statfs":{"total":773094113280,"available":772852195328,"internally_reserved":0,"allocated":3481600,"data_stored":2008928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":22,"apply_latency_ms":22,"commit_latency_ns":22000000,"apply_latency_ns":22000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001173"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:43.907028+0000","last_change":"2024-03-16T20:45:43.907028+0000","last_active":"2024-03-16T20:45:43.907028+0000","last_peered":"2024-03-16T20:45:43.907028+0000","last_clean":"2024-03-16T20:45:43.907028+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:43.907028+0000","last_undegraded":"2024-03-16T20:45:43.907028+0000","last_fullsized":"2024-03-16T20:45:43.907028+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-03-16T20:45:42.898452+0000","last_change":"2024-03-16T20:45:42.898452+0000","last_active":"2024-03-16T20:45:42.898452+0000","last_peered":"2024-03-16T20:45:42.898452+0000","last_clean":"2024-03-16T20:45:42.898452+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-03-16T20:45:42.898452+0000","last_undegraded":"2024-03-16T20:45:42.898452+0000","last_fullsized":"2024-03-16T20:45:42.898452+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":179,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-03-16T20:45:39.263313+0000","last_change":"2024-03-16T20:44:54.293418+0000","last_active":"2024-03-16T20:45:39.263313+0000","last_peered":"2024-03-16T20:45:39.263313+0000","last_clean":"2024-03-16T20:45:39.263313+0000","last_became_active":"2024-03-16T20:44:54.291347+0000","last_became_peered":"2024-03-16T20:44:54.291347+0000","last_unstale":"2024-03-16T20:45:39.263313+0000","last_undegraded":"2024-03-16T20:45:39.263313+0000","last_fullsized":"2024-03-16T20:45:39.263313+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_clean_scrub_stamp":"2024-03-16T20:44:53.077639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:29:18.318494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":880640,"data_stored":693892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31468,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":286720,"data_stored":103524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31468,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":286720,"data_stored":103524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31468,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":286720,"data_stored":103524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574861,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":103524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27308,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":286720,"data_stored":103524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27304,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":286720,"data_stored":103524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":880640,"data_stored":693892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T20:45:48.445 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T20:45:48.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.599+0000 7f727ebaf640 1 Processor -- start 2024-03-16T20:45:48.605 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.599+0000 7f727ebaf640 1 -- start start 2024-03-16T20:45:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f727ebaf640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f72781472b0 0x7f727814a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f727ebaf640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7278147af0 0x7f72780b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f727ebaf640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7278072680 con 0x7f72780b3650 2024-03-16T20:45:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f727ebaf640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72780b45b0 con 0x7f7278147af0 2024-03-16T20:45:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f727ebaf640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72780b5760 con 0x7f72781472b0 2024-03-16T20:45:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f7277fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7278147af0 0x7f72780b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f7277fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7278147af0 0x7f72780b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49090/0 (socket says 172.21.15.122:49090) 2024-03-16T20:45:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f7277fff640 1 -- 172.21.15.122:0/2473200081 learned_addr learned my addr 172.21.15.122:0/2473200081 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f727c94b640 1 --2- 172.21.15.122:0/2473200081 >> v2:172.21.15.195:3300/0 conn(0x7f72781472b0 0x7f727814a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f7277fff640 1 -- 172.21.15.122:0/2473200081 >> v2:172.21.15.195:3300/0 conn(0x7f72781472b0 msgr2=0x7f727814a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f7277fff640 1 --2- 172.21.15.122:0/2473200081 >> v2:172.21.15.195:3300/0 conn(0x7f72781472b0 0x7f727814a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f7277fff640 1 -- 172.21.15.122:0/2473200081 >> v1:172.21.15.195:6789/0 conn(0x7f72780b3650 legacy=0x7f727806af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f7277fff640 1 -- 172.21.15.122:0/2473200081 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7278147f50 con 0x7f7278147af0 2024-03-16T20:45:48.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f727c94b640 1 --2- 172.21.15.122:0/2473200081 >> v2:172.21.15.195:3300/0 conn(0x7f72781472b0 0x7f727814a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f7277fff640 1 --2- 172.21.15.122:0/2473200081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7278147af0 0x7f72780b3110 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f72780b6eb0 tx=0x7f726001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29abab16642ca60c server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:48.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f72777fe640 1 -- 172.21.15.122:0/2473200081 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72600ab600 con 0x7f7278147af0 2024-03-16T20:45:48.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f72777fe640 1 -- 172.21.15.122:0/2473200081 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72600a7050 con 0x7f7278147af0 2024-03-16T20:45:48.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.603+0000 7f72777fe640 1 -- 172.21.15.122:0/2473200081 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72600ab980 con 0x7f7278147af0 2024-03-16T20:45:48.610 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 -- 172.21.15.122:0/2473200081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7278147af0 msgr2=0x7f72780b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 --2- 172.21.15.122:0/2473200081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7278147af0 0x7f72780b3110 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f72780b6eb0 tx=0x7f726001c450 comp rx=0 tx=0).stop 2024-03-16T20:45:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 -- 172.21.15.122:0/2473200081 shutdown_connections 2024-03-16T20:45:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 --2- 172.21.15.122:0/2473200081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7278147af0 0x7f72780b3110 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 --2- 172.21.15.122:0/2473200081 >> v2:172.21.15.195:3300/0 conn(0x7f72781472b0 0x7f727814a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 -- 172.21.15.122:0/2473200081 >> 172.21.15.122:0/2473200081 conn(0x7f72780b62a0 msgr2=0x7f72780b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 -- 172.21.15.122:0/2473200081 shutdown_connections 2024-03-16T20:45:48.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 -- 172.21.15.122:0/2473200081 wait complete. 2024-03-16T20:45:48.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 Processor -- start 2024-03-16T20:45:48.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 -- start start 2024-03-16T20:45:48.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72780b3650 0x7f72780b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:48.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72781472b0 0x7f727813c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:48.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7278147af0 0x7f727813a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:48.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727813ce70 con 0x7f72781472b0 2024-03-16T20:45:48.613 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7278140160 con 0x7f7278147af0 2024-03-16T20:45:48.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727813c380 con 0x7f72780b3650 2024-03-16T20:45:48.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f7277fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72781472b0 0x7f727813c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:48.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f7277fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72781472b0 0x7f727813c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49096/0 (socket says 172.21.15.122:49096) 2024-03-16T20:45:48.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f7277fff640 1 -- 172.21.15.122:0/3319100543 learned_addr learned my addr 172.21.15.122:0/3319100543 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:48.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727c94b640 1 --2- 172.21.15.122:0/3319100543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72780b3650 0x7f72780b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:48.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727d14c640 1 --2- 172.21.15.122:0/3319100543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7278147af0 0x7f727813a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:48.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f7277fff640 1 -- 172.21.15.122:0/3319100543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72780b3650 msgr2=0x7f72780b2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:48.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f7277fff640 1 --2- 172.21.15.122:0/3319100543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72780b3650 0x7f72780b2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f7277fff640 1 -- 172.21.15.122:0/3319100543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7278147af0 msgr2=0x7f727813a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:48.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f7277fff640 1 --2- 172.21.15.122:0/3319100543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7278147af0 0x7f727813a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f7277fff640 1 -- 172.21.15.122:0/3319100543 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72600094d0 con 0x7f72781472b0 2024-03-16T20:45:48.614 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f7277fff640 1 --2- 172.21.15.122:0/3319100543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72781472b0 0x7f727813c930 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f72600ae040 tx=0x7f726001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:48.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f72757fa640 1 -- 172.21.15.122:0/3319100543 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72600b6830 con 0x7f72781472b0 2024-03-16T20:45:48.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f72757fa640 1 -- 172.21.15.122:0/3319100543 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72600b69b0 con 0x7f72781472b0 2024-03-16T20:45:48.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f72757fa640 1 -- 172.21.15.122:0/3319100543 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72600b6d60 con 0x7f72781472b0 2024-03-16T20:45:48.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 -- 172.21.15.122:0/3319100543 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727813d0f0 con 0x7f72781472b0 2024-03-16T20:45:48.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.607+0000 7f727ebaf640 1 -- 172.21.15.122:0/3319100543 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727813b650 con 0x7f72781472b0 2024-03-16T20:45:48.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.611+0000 7f727ebaf640 1 -- 172.21.15.122:0/3319100543 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7278075280 con 0x7f72781472b0 2024-03-16T20:45:48.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.611+0000 7f72757fa640 1 -- 172.21.15.122:0/3319100543 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f72600b2040 con 0x7f72781472b0 2024-03-16T20:45:48.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.611+0000 7f72757fa640 1 --2- 172.21.15.122:0/3319100543 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7258066e40 0x7f7258069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:48.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.611+0000 7f72757fa640 1 -- 172.21.15.122:0/3319100543 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f726011f0b0 con 0x7f72781472b0 2024-03-16T20:45:48.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.611+0000 7f727c94b640 1 --2- 172.21.15.122:0/3319100543 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7258066e40 0x7f7258069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:48.616 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.611+0000 7f727c94b640 1 --2- 172.21.15.122:0/3319100543 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7258066e40 0x7f7258069300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f727813d990 tx=0x7f7268008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:48.619 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.615+0000 7f72757fa640 1 -- 172.21.15.122:0/3319100543 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f72600ebf30 con 0x7f72781472b0 2024-03-16T20:45:48.892 INFO:tasks.ceph.mds.a.smithi122.stderr:2024-03-16T20:45:48.887+0000 7fe85768f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-16T20:45:48.908 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.903+0000 7f727ebaf640 1 -- 172.21.15.122:0/3319100543 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f72781413d0 con 0x7f7258066e40 2024-03-16T20:45:48.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.919+0000 7f72757fa640 1 -- 172.21.15.122:0/3319100543 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251945 (secure 0 0 0) 0x7f72781413d0 con 0x7f7258066e40 2024-03-16T20:45:48.928 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:48.929 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-03-16T20:45:48.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.923+0000 7f727ebaf640 1 -- 172.21.15.122:0/3319100543 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7258066e40 msgr2=0x7f7258069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:48.930 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.923+0000 7f727ebaf640 1 --2- 172.21.15.122:0/3319100543 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7258066e40 0x7f7258069300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f727813d990 tx=0x7f7268008040 comp rx=0 tx=0).stop 2024-03-16T20:45:48.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.923+0000 7f727ebaf640 1 -- 172.21.15.122:0/3319100543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72781472b0 msgr2=0x7f727813c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:48.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.923+0000 7f727ebaf640 1 --2- 172.21.15.122:0/3319100543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72781472b0 0x7f727813c930 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f72600ae040 tx=0x7f726001ac90 comp rx=0 tx=0).stop 2024-03-16T20:45:48.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.923+0000 7f727ebaf640 1 -- 172.21.15.122:0/3319100543 shutdown_connections 2024-03-16T20:45:48.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.923+0000 7f727ebaf640 1 --2- 172.21.15.122:0/3319100543 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7258066e40 0x7f7258069300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.923+0000 7f727ebaf640 1 --2- 172.21.15.122:0/3319100543 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7278147af0 0x7f727813a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.923+0000 7f727ebaf640 1 --2- 172.21.15.122:0/3319100543 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f72781472b0 0x7f727813c930 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.923+0000 7f727ebaf640 1 --2- 172.21.15.122:0/3319100543 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f72780b3650 0x7f72780b2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:48.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.923+0000 7f727ebaf640 1 -- 172.21.15.122:0/3319100543 >> 172.21.15.122:0/3319100543 conn(0x7f72780b62a0 msgr2=0x7f72780719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:48.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.923+0000 7f727ebaf640 1 -- 172.21.15.122:0/3319100543 shutdown_connections 2024-03-16T20:45:48.931 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:48.923+0000 7f727ebaf640 1 -- 172.21.15.122:0/3319100543 wait complete. 2024-03-16T20:45:48.944 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-03-16T20:45:48.591007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236664,"kb_used_data":3824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738056,"statfs":{"total":773094113280,"available":772851769344,"internally_reserved":0,"allocated":3915776,"data_stored":2202420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.682889"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268260+0000","last_change":"2024-03-16T20:45:44.977479+0000","last_active":"2024-03-16T20:45:46.268260+0000","last_peered":"2024-03-16T20:45:46.268260+0000","last_clean":"2024-03-16T20:45:46.268260+0000","last_became_active":"2024-03-16T20:45:44.974377+0000","last_became_peered":"2024-03-16T20:45:44.974377+0000","last_unstale":"2024-03-16T20:45:46.268260+0000","last_undegraded":"2024-03-16T20:45:46.268260+0000","last_fullsized":"2024-03-16T20:45:46.268260+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:40:04.897827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266582+0000","last_change":"2024-03-16T20:45:44.969210+0000","last_active":"2024-03-16T20:45:46.266582+0000","last_peered":"2024-03-16T20:45:46.266582+0000","last_clean":"2024-03-16T20:45:46.266582+0000","last_became_active":"2024-03-16T20:45:44.968476+0000","last_became_peered":"2024-03-16T20:45:44.968476+0000","last_unstale":"2024-03-16T20:45:46.266582+0000","last_undegraded":"2024-03-16T20:45:46.266582+0000","last_fullsized":"2024-03-16T20:45:46.266582+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:54:20.920066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957286+0000","last_change":"2024-03-16T20:45:44.986911+0000","last_active":"2024-03-16T20:45:45.957286+0000","last_peered":"2024-03-16T20:45:45.957286+0000","last_clean":"2024-03-16T20:45:45.957286+0000","last_became_active":"2024-03-16T20:45:44.984194+0000","last_became_peered":"2024-03-16T20:45:44.984194+0000","last_unstale":"2024-03-16T20:45:45.957286+0000","last_undegraded":"2024-03-16T20:45:45.957286+0000","last_fullsized":"2024-03-16T20:45:45.957286+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:49.546012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957075+0000","last_change":"2024-03-16T20:45:45.002983+0000","last_active":"2024-03-16T20:45:45.957075+0000","last_peered":"2024-03-16T20:45:45.957075+0000","last_clean":"2024-03-16T20:45:45.957075+0000","last_became_active":"2024-03-16T20:45:45.002338+0000","last_became_peered":"2024-03-16T20:45:45.002338+0000","last_unstale":"2024-03-16T20:45:45.957075+0000","last_undegraded":"2024-03-16T20:45:45.957075+0000","last_fullsized":"2024-03-16T20:45:45.957075+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:05:57.364887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267523+0000","last_change":"2024-03-16T20:45:44.980896+0000","last_active":"2024-03-16T20:45:46.267523+0000","last_peered":"2024-03-16T20:45:46.267523+0000","last_clean":"2024-03-16T20:45:46.267523+0000","last_became_active":"2024-03-16T20:45:44.979594+0000","last_became_peered":"2024-03-16T20:45:44.979594+0000","last_unstale":"2024-03-16T20:45:46.267523+0000","last_undegraded":"2024-03-16T20:45:46.267523+0000","last_fullsized":"2024-03-16T20:45:46.267523+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:40:57.340772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266581+0000","last_change":"2024-03-16T20:45:44.975175+0000","last_active":"2024-03-16T20:45:46.266581+0000","last_peered":"2024-03-16T20:45:46.266581+0000","last_clean":"2024-03-16T20:45:46.266581+0000","last_became_active":"2024-03-16T20:45:44.974424+0000","last_became_peered":"2024-03-16T20:45:44.974424+0000","last_unstale":"2024-03-16T20:45:46.266581+0000","last_undegraded":"2024-03-16T20:45:46.266581+0000","last_fullsized":"2024-03-16T20:45:46.266581+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:08:10.036696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269399+0000","last_change":"2024-03-16T20:45:44.979934+0000","last_active":"2024-03-16T20:45:46.269399+0000","last_peered":"2024-03-16T20:45:46.269399+0000","last_clean":"2024-03-16T20:45:46.269399+0000","last_became_active":"2024-03-16T20:45:44.979060+0000","last_became_peered":"2024-03-16T20:45:44.979060+0000","last_unstale":"2024-03-16T20:45:46.269399+0000","last_undegraded":"2024-03-16T20:45:46.269399+0000","last_fullsized":"2024-03-16T20:45:46.269399+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:54:20.920066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268330+0000","last_change":"2024-03-16T20:45:44.987586+0000","last_active":"2024-03-16T20:45:46.268330+0000","last_peered":"2024-03-16T20:45:46.268330+0000","last_clean":"2024-03-16T20:45:46.268330+0000","last_became_active":"2024-03-16T20:45:44.986860+0000","last_became_peered":"2024-03-16T20:45:44.986860+0000","last_unstale":"2024-03-16T20:45:46.268330+0000","last_undegraded":"2024-03-16T20:45:46.268330+0000","last_fullsized":"2024-03-16T20:45:46.268330+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:31:14.070831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956478+0000","last_change":"2024-03-16T20:45:44.997710+0000","last_active":"2024-03-16T20:45:45.956478+0000","last_peered":"2024-03-16T20:45:45.956478+0000","last_clean":"2024-03-16T20:45:45.956478+0000","last_became_active":"2024-03-16T20:45:44.996999+0000","last_became_peered":"2024-03-16T20:45:44.996999+0000","last_unstale":"2024-03-16T20:45:45.956478+0000","last_undegraded":"2024-03-16T20:45:45.956478+0000","last_fullsized":"2024-03-16T20:45:45.956478+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:40:04.897827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957631+0000","last_change":"2024-03-16T20:45:44.971505+0000","last_active":"2024-03-16T20:45:45.957631+0000","last_peered":"2024-03-16T20:45:45.957631+0000","last_clean":"2024-03-16T20:45:45.957631+0000","last_became_active":"2024-03-16T20:45:44.970737+0000","last_became_peered":"2024-03-16T20:45:44.970737+0000","last_unstale":"2024-03-16T20:45:45.957631+0000","last_undegraded":"2024-03-16T20:45:45.957631+0000","last_fullsized":"2024-03-16T20:45:45.957631+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:06:36.112288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269089+0000","last_change":"2024-03-16T20:45:44.988896+0000","last_active":"2024-03-16T20:45:46.269089+0000","last_peered":"2024-03-16T20:45:46.269089+0000","last_clean":"2024-03-16T20:45:46.269089+0000","last_became_active":"2024-03-16T20:45:44.988250+0000","last_became_peered":"2024-03-16T20:45:44.988250+0000","last_unstale":"2024-03-16T20:45:46.269089+0000","last_undegraded":"2024-03-16T20:45:46.269089+0000","last_fullsized":"2024-03-16T20:45:46.269089+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:57:53.975463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956681+0000","last_change":"2024-03-16T20:45:44.969082+0000","last_active":"2024-03-16T20:45:45.956681+0000","last_peered":"2024-03-16T20:45:45.956681+0000","last_clean":"2024-03-16T20:45:45.956681+0000","last_became_active":"2024-03-16T20:45:44.968282+0000","last_became_peered":"2024-03-16T20:45:44.968282+0000","last_unstale":"2024-03-16T20:45:45.956681+0000","last_undegraded":"2024-03-16T20:45:45.956681+0000","last_fullsized":"2024-03-16T20:45:45.956681+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:05:57.364887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267692+0000","last_change":"2024-03-16T20:45:44.984346+0000","last_active":"2024-03-16T20:45:46.267692+0000","last_peered":"2024-03-16T20:45:46.267692+0000","last_clean":"2024-03-16T20:45:46.267692+0000","last_became_active":"2024-03-16T20:45:44.983837+0000","last_became_peered":"2024-03-16T20:45:44.983837+0000","last_unstale":"2024-03-16T20:45:46.267692+0000","last_undegraded":"2024-03-16T20:45:46.267692+0000","last_fullsized":"2024-03-16T20:45:46.267692+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:46:10.557343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266735+0000","last_change":"2024-03-16T20:45:44.969608+0000","last_active":"2024-03-16T20:45:46.266735+0000","last_peered":"2024-03-16T20:45:46.266735+0000","last_clean":"2024-03-16T20:45:46.266735+0000","last_became_active":"2024-03-16T20:45:44.968876+0000","last_became_peered":"2024-03-16T20:45:44.968876+0000","last_unstale":"2024-03-16T20:45:46.266735+0000","last_undegraded":"2024-03-16T20:45:46.266735+0000","last_fullsized":"2024-03-16T20:45:46.266735+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:20:45.687785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.955788+0000","last_change":"2024-03-16T20:45:44.975592+0000","last_active":"2024-03-16T20:45:45.955788+0000","last_peered":"2024-03-16T20:45:45.955788+0000","last_clean":"2024-03-16T20:45:45.955788+0000","last_became_active":"2024-03-16T20:45:44.973733+0000","last_became_peered":"2024-03-16T20:45:44.973733+0000","last_unstale":"2024-03-16T20:45:45.955788+0000","last_undegraded":"2024-03-16T20:45:45.955788+0000","last_fullsized":"2024-03-16T20:45:45.955788+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:31:14.070831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269494+0000","last_change":"2024-03-16T20:45:44.976648+0000","last_active":"2024-03-16T20:45:46.269494+0000","last_peered":"2024-03-16T20:45:46.269494+0000","last_clean":"2024-03-16T20:45:46.269494+0000","last_became_active":"2024-03-16T20:45:44.975885+0000","last_became_peered":"2024-03-16T20:45:44.975885+0000","last_unstale":"2024-03-16T20:45:46.269494+0000","last_undegraded":"2024-03-16T20:45:46.269494+0000","last_fullsized":"2024-03-16T20:45:46.269494+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:08:10.036696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268613+0000","last_change":"2024-03-16T20:45:44.983529+0000","last_active":"2024-03-16T20:45:46.268613+0000","last_peered":"2024-03-16T20:45:46.268613+0000","last_clean":"2024-03-16T20:45:46.268613+0000","last_became_active":"2024-03-16T20:45:44.982855+0000","last_became_peered":"2024-03-16T20:45:44.982855+0000","last_unstale":"2024-03-16T20:45:46.268613+0000","last_undegraded":"2024-03-16T20:45:46.268613+0000","last_fullsized":"2024-03-16T20:45:46.268613+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:45:12.371596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266734+0000","last_change":"2024-03-16T20:45:44.986663+0000","last_active":"2024-03-16T20:45:46.266734+0000","last_peered":"2024-03-16T20:45:46.266734+0000","last_clean":"2024-03-16T20:45:46.266734+0000","last_became_active":"2024-03-16T20:45:44.985802+0000","last_became_peered":"2024-03-16T20:45:44.985802+0000","last_unstale":"2024-03-16T20:45:46.266734+0000","last_undegraded":"2024-03-16T20:45:46.266734+0000","last_fullsized":"2024-03-16T20:45:46.266734+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:29:17.211554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266824+0000","last_change":"2024-03-16T20:45:44.971503+0000","last_active":"2024-03-16T20:45:46.266824+0000","last_peered":"2024-03-16T20:45:46.266824+0000","last_clean":"2024-03-16T20:45:46.266824+0000","last_became_active":"2024-03-16T20:45:44.970764+0000","last_became_peered":"2024-03-16T20:45:44.970764+0000","last_unstale":"2024-03-16T20:45:46.266824+0000","last_undegraded":"2024-03-16T20:45:46.266824+0000","last_fullsized":"2024-03-16T20:45:46.266824+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:51:07.991442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267798+0000","last_change":"2024-03-16T20:45:44.981615+0000","last_active":"2024-03-16T20:45:46.267798+0000","last_peered":"2024-03-16T20:45:46.267798+0000","last_clean":"2024-03-16T20:45:46.267798+0000","last_became_active":"2024-03-16T20:45:44.980565+0000","last_became_peered":"2024-03-16T20:45:44.980565+0000","last_unstale":"2024-03-16T20:45:46.267798+0000","last_undegraded":"2024-03-16T20:45:46.267798+0000","last_fullsized":"2024-03-16T20:45:46.267798+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:42:10.851650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957994+0000","last_change":"2024-03-16T20:45:45.001667+0000","last_active":"2024-03-16T20:45:45.957994+0000","last_peered":"2024-03-16T20:45:45.957994+0000","last_clean":"2024-03-16T20:45:45.957994+0000","last_became_active":"2024-03-16T20:45:44.999133+0000","last_became_peered":"2024-03-16T20:45:44.999133+0000","last_unstale":"2024-03-16T20:45:45.957994+0000","last_undegraded":"2024-03-16T20:45:45.957994+0000","last_fullsized":"2024-03-16T20:45:45.957994+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:45:23.479155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956254+0000","last_change":"2024-03-16T20:45:44.996905+0000","last_active":"2024-03-16T20:45:45.956254+0000","last_peered":"2024-03-16T20:45:45.956254+0000","last_clean":"2024-03-16T20:45:45.956254+0000","last_became_active":"2024-03-16T20:45:44.996182+0000","last_became_peered":"2024-03-16T20:45:44.996182+0000","last_unstale":"2024-03-16T20:45:45.956254+0000","last_undegraded":"2024-03-16T20:45:45.956254+0000","last_fullsized":"2024-03-16T20:45:45.956254+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:41:27.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005074+0000","last_change":"2024-03-16T20:45:44.972167+0000","last_active":"2024-03-16T20:45:46.005074+0000","last_peered":"2024-03-16T20:45:46.005074+0000","last_clean":"2024-03-16T20:45:46.005074+0000","last_became_active":"2024-03-16T20:45:44.971283+0000","last_became_peered":"2024-03-16T20:45:44.971283+0000","last_unstale":"2024-03-16T20:45:46.005074+0000","last_undegraded":"2024-03-16T20:45:46.005074+0000","last_fullsized":"2024-03-16T20:45:46.005074+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:41:27.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.955247+0000","last_change":"2024-03-16T20:45:44.977536+0000","last_active":"2024-03-16T20:45:45.955247+0000","last_peered":"2024-03-16T20:45:45.955247+0000","last_clean":"2024-03-16T20:45:45.955247+0000","last_became_active":"2024-03-16T20:45:44.976794+0000","last_became_peered":"2024-03-16T20:45:44.976794+0000","last_unstale":"2024-03-16T20:45:45.955247+0000","last_undegraded":"2024-03-16T20:45:45.955247+0000","last_fullsized":"2024-03-16T20:45:45.955247+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:35:59.048311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957835+0000","last_change":"2024-03-16T20:45:44.976909+0000","last_active":"2024-03-16T20:45:45.957835+0000","last_peered":"2024-03-16T20:45:45.957835+0000","last_clean":"2024-03-16T20:45:45.957835+0000","last_became_active":"2024-03-16T20:45:44.976127+0000","last_became_peered":"2024-03-16T20:45:44.976127+0000","last_unstale":"2024-03-16T20:45:45.957835+0000","last_undegraded":"2024-03-16T20:45:45.957835+0000","last_fullsized":"2024-03-16T20:45:45.957835+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:45:12.371596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266763+0000","last_change":"2024-03-16T20:45:44.978761+0000","last_active":"2024-03-16T20:45:46.266763+0000","last_peered":"2024-03-16T20:45:46.266763+0000","last_clean":"2024-03-16T20:45:46.266763+0000","last_became_active":"2024-03-16T20:45:44.974520+0000","last_became_peered":"2024-03-16T20:45:44.974520+0000","last_unstale":"2024-03-16T20:45:46.266763+0000","last_undegraded":"2024-03-16T20:45:46.266763+0000","last_fullsized":"2024-03-16T20:45:46.266763+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:14:14.791268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957901+0000","last_change":"2024-03-16T20:45:45.004121+0000","last_active":"2024-03-16T20:45:45.957901+0000","last_peered":"2024-03-16T20:45:45.957901+0000","last_clean":"2024-03-16T20:45:45.957901+0000","last_became_active":"2024-03-16T20:45:45.003251+0000","last_became_peered":"2024-03-16T20:45:45.003251+0000","last_unstale":"2024-03-16T20:45:45.957901+0000","last_undegraded":"2024-03-16T20:45:45.957901+0000","last_fullsized":"2024-03-16T20:45:45.957901+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:39:41.263058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957018+0000","last_change":"2024-03-16T20:45:44.964295+0000","last_active":"2024-03-16T20:45:45.957018+0000","last_peered":"2024-03-16T20:45:45.957018+0000","last_clean":"2024-03-16T20:45:45.957018+0000","last_became_active":"2024-03-16T20:45:44.963502+0000","last_became_peered":"2024-03-16T20:45:44.963502+0000","last_unstale":"2024-03-16T20:45:45.957018+0000","last_undegraded":"2024-03-16T20:45:45.957018+0000","last_fullsized":"2024-03-16T20:45:45.957018+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:39:41.263058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957545+0000","last_change":"2024-03-16T20:45:45.004296+0000","last_active":"2024-03-16T20:45:45.957545+0000","last_peered":"2024-03-16T20:45:45.957545+0000","last_clean":"2024-03-16T20:45:45.957545+0000","last_became_active":"2024-03-16T20:45:44.997947+0000","last_became_peered":"2024-03-16T20:45:44.997947+0000","last_unstale":"2024-03-16T20:45:45.957545+0000","last_undegraded":"2024-03-16T20:45:45.957545+0000","last_fullsized":"2024-03-16T20:45:45.957545+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:25:32.739512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268596+0000","last_change":"2024-03-16T20:45:44.983629+0000","last_active":"2024-03-16T20:45:46.268596+0000","last_peered":"2024-03-16T20:45:46.268596+0000","last_clean":"2024-03-16T20:45:46.268596+0000","last_became_active":"2024-03-16T20:45:44.979161+0000","last_became_peered":"2024-03-16T20:45:44.979161+0000","last_unstale":"2024-03-16T20:45:46.268596+0000","last_undegraded":"2024-03-16T20:45:46.268596+0000","last_fullsized":"2024-03-16T20:45:46.268596+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:06:36.112288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267550+0000","last_change":"2024-03-16T20:45:44.977815+0000","last_active":"2024-03-16T20:45:46.267550+0000","last_peered":"2024-03-16T20:45:46.267550+0000","last_clean":"2024-03-16T20:45:46.267550+0000","last_became_active":"2024-03-16T20:45:44.975655+0000","last_became_peered":"2024-03-16T20:45:44.975655+0000","last_unstale":"2024-03-16T20:45:46.267550+0000","last_undegraded":"2024-03-16T20:45:46.267550+0000","last_fullsized":"2024-03-16T20:45:46.267550+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:30:32.602019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268765+0000","last_change":"2024-03-16T20:45:45.003806+0000","last_active":"2024-03-16T20:45:46.268765+0000","last_peered":"2024-03-16T20:45:46.268765+0000","last_clean":"2024-03-16T20:45:46.268765+0000","last_became_active":"2024-03-16T20:45:45.002935+0000","last_became_peered":"2024-03-16T20:45:45.002935+0000","last_unstale":"2024-03-16T20:45:46.268765+0000","last_undegraded":"2024-03-16T20:45:46.268765+0000","last_fullsized":"2024-03-16T20:45:46.268765+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:49.546012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005668+0000","last_change":"2024-03-16T20:45:44.977311+0000","last_active":"2024-03-16T20:45:46.005668+0000","last_peered":"2024-03-16T20:45:46.005668+0000","last_clean":"2024-03-16T20:45:46.005668+0000","last_became_active":"2024-03-16T20:45:44.975825+0000","last_became_peered":"2024-03-16T20:45:44.975825+0000","last_unstale":"2024-03-16T20:45:46.005668+0000","last_undegraded":"2024-03-16T20:45:46.005668+0000","last_fullsized":"2024-03-16T20:45:46.005668+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:25:32.739512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"29'4","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269202+0000","last_change":"2024-03-16T20:45:43.956395+0000","last_active":"2024-03-16T20:45:46.269202+0000","last_peered":"2024-03-16T20:45:46.269202+0000","last_clean":"2024-03-16T20:45:46.269202+0000","last_became_active":"2024-03-16T20:45:43.955548+0000","last_became_peered":"2024-03-16T20:45:43.955548+0000","last_unstale":"2024-03-16T20:45:46.269202+0000","last_undegraded":"2024-03-16T20:45:46.269202+0000","last_fullsized":"2024-03-16T20:45:46.269202+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:29:54.412142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268904+0000","last_change":"2024-03-16T20:45:45.001424+0000","last_active":"2024-03-16T20:45:46.268904+0000","last_peered":"2024-03-16T20:45:46.268904+0000","last_clean":"2024-03-16T20:45:46.268904+0000","last_became_active":"2024-03-16T20:45:45.000845+0000","last_became_peered":"2024-03-16T20:45:45.000845+0000","last_unstale":"2024-03-16T20:45:46.268904+0000","last_undegraded":"2024-03-16T20:45:46.268904+0000","last_fullsized":"2024-03-16T20:45:46.268904+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:57:51.937834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005625+0000","last_change":"2024-03-16T20:45:43.941139+0000","last_active":"2024-03-16T20:45:46.005625+0000","last_peered":"2024-03-16T20:45:46.005625+0000","last_clean":"2024-03-16T20:45:46.005625+0000","last_became_active":"2024-03-16T20:45:43.940339+0000","last_became_peered":"2024-03-16T20:45:43.940339+0000","last_unstale":"2024-03-16T20:45:46.005625+0000","last_undegraded":"2024-03-16T20:45:46.005625+0000","last_fullsized":"2024-03-16T20:45:46.005625+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:11:09.879241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956868+0000","last_change":"2024-03-16T20:45:44.999396+0000","last_active":"2024-03-16T20:45:45.956868+0000","last_peered":"2024-03-16T20:45:45.956868+0000","last_clean":"2024-03-16T20:45:45.956868+0000","last_became_active":"2024-03-16T20:45:44.997352+0000","last_became_peered":"2024-03-16T20:45:44.997352+0000","last_unstale":"2024-03-16T20:45:45.956868+0000","last_undegraded":"2024-03-16T20:45:45.956868+0000","last_fullsized":"2024-03-16T20:45:45.956868+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:57:51.937834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266929+0000","last_change":"2024-03-16T20:45:43.932198+0000","last_active":"2024-03-16T20:45:46.266929+0000","last_peered":"2024-03-16T20:45:46.266929+0000","last_clean":"2024-03-16T20:45:46.266929+0000","last_became_active":"2024-03-16T20:45:43.931513+0000","last_became_peered":"2024-03-16T20:45:43.931513+0000","last_unstale":"2024-03-16T20:45:46.266929+0000","last_undegraded":"2024-03-16T20:45:46.266929+0000","last_fullsized":"2024-03-16T20:45:46.266929+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:30:31.593443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957955+0000","last_change":"2024-03-16T20:45:45.003785+0000","last_active":"2024-03-16T20:45:45.957955+0000","last_peered":"2024-03-16T20:45:45.957955+0000","last_clean":"2024-03-16T20:45:45.957955+0000","last_became_active":"2024-03-16T20:45:45.003497+0000","last_became_peered":"2024-03-16T20:45:45.003497+0000","last_unstale":"2024-03-16T20:45:45.957955+0000","last_undegraded":"2024-03-16T20:45:45.957955+0000","last_fullsized":"2024-03-16T20:45:45.957955+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:49.546012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267880+0000","last_change":"2024-03-16T20:45:43.929779+0000","last_active":"2024-03-16T20:45:46.267880+0000","last_peered":"2024-03-16T20:45:46.267880+0000","last_clean":"2024-03-16T20:45:46.267880+0000","last_became_active":"2024-03-16T20:45:43.929496+0000","last_became_peered":"2024-03-16T20:45:43.929496+0000","last_unstale":"2024-03-16T20:45:46.267880+0000","last_undegraded":"2024-03-16T20:45:46.267880+0000","last_fullsized":"2024-03-16T20:45:46.267880+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:35:46.461735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268734+0000","last_change":"2024-03-16T20:45:45.000918+0000","last_active":"2024-03-16T20:45:46.268734+0000","last_peered":"2024-03-16T20:45:46.268734+0000","last_clean":"2024-03-16T20:45:46.268734+0000","last_became_active":"2024-03-16T20:45:45.000298+0000","last_became_peered":"2024-03-16T20:45:45.000298+0000","last_unstale":"2024-03-16T20:45:46.268734+0000","last_undegraded":"2024-03-16T20:45:46.268734+0000","last_fullsized":"2024-03-16T20:45:46.268734+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:39:41.263058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957811+0000","last_change":"2024-03-16T20:45:43.942190+0000","last_active":"2024-03-16T20:45:45.957811+0000","last_peered":"2024-03-16T20:45:45.957811+0000","last_clean":"2024-03-16T20:45:45.957811+0000","last_became_active":"2024-03-16T20:45:43.938832+0000","last_became_peered":"2024-03-16T20:45:43.938832+0000","last_unstale":"2024-03-16T20:45:45.957811+0000","last_undegraded":"2024-03-16T20:45:45.957811+0000","last_fullsized":"2024-03-16T20:45:45.957811+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:04:09.454448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957353+0000","last_change":"2024-03-16T20:45:44.966859+0000","last_active":"2024-03-16T20:45:45.957353+0000","last_peered":"2024-03-16T20:45:45.957353+0000","last_clean":"2024-03-16T20:45:45.957353+0000","last_became_active":"2024-03-16T20:45:44.966087+0000","last_became_peered":"2024-03-16T20:45:44.966087+0000","last_unstale":"2024-03-16T20:45:45.957353+0000","last_undegraded":"2024-03-16T20:45:45.957353+0000","last_fullsized":"2024-03-16T20:45:45.957353+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:57:53.975463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956556+0000","last_change":"2024-03-16T20:45:43.935054+0000","last_active":"2024-03-16T20:45:45.956556+0000","last_peered":"2024-03-16T20:45:45.956556+0000","last_clean":"2024-03-16T20:45:45.956556+0000","last_became_active":"2024-03-16T20:45:43.933153+0000","last_became_peered":"2024-03-16T20:45:43.933153+0000","last_unstale":"2024-03-16T20:45:45.956556+0000","last_undegraded":"2024-03-16T20:45:45.956556+0000","last_fullsized":"2024-03-16T20:45:45.956556+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:05:20.864966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.955670+0000","last_change":"2024-03-16T20:45:44.997096+0000","last_active":"2024-03-16T20:45:45.955670+0000","last_peered":"2024-03-16T20:45:45.955670+0000","last_clean":"2024-03-16T20:45:45.955670+0000","last_became_active":"2024-03-16T20:45:44.996219+0000","last_became_peered":"2024-03-16T20:45:44.996219+0000","last_unstale":"2024-03-16T20:45:45.955670+0000","last_undegraded":"2024-03-16T20:45:45.955670+0000","last_fullsized":"2024-03-16T20:45:45.955670+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:59:40.370844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266166+0000","last_change":"2024-03-16T20:45:43.929969+0000","last_active":"2024-03-16T20:45:46.266166+0000","last_peered":"2024-03-16T20:45:46.266166+0000","last_clean":"2024-03-16T20:45:46.266166+0000","last_became_active":"2024-03-16T20:45:43.929709+0000","last_became_peered":"2024-03-16T20:45:43.929709+0000","last_unstale":"2024-03-16T20:45:46.266166+0000","last_undegraded":"2024-03-16T20:45:46.266166+0000","last_fullsized":"2024-03-16T20:45:46.266166+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:07:47.365995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956485+0000","last_change":"2024-03-16T20:45:44.969933+0000","last_active":"2024-03-16T20:45:45.956485+0000","last_peered":"2024-03-16T20:45:45.956485+0000","last_clean":"2024-03-16T20:45:45.956485+0000","last_became_active":"2024-03-16T20:45:44.969188+0000","last_became_peered":"2024-03-16T20:45:44.969188+0000","last_unstale":"2024-03-16T20:45:45.956485+0000","last_undegraded":"2024-03-16T20:45:45.956485+0000","last_fullsized":"2024-03-16T20:45:45.956485+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:35:59.048311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956640+0000","last_change":"2024-03-16T20:45:43.937169+0000","last_active":"2024-03-16T20:45:45.956640+0000","last_peered":"2024-03-16T20:45:45.956640+0000","last_clean":"2024-03-16T20:45:45.956640+0000","last_became_active":"2024-03-16T20:45:43.936408+0000","last_became_peered":"2024-03-16T20:45:43.936408+0000","last_unstale":"2024-03-16T20:45:45.956640+0000","last_undegraded":"2024-03-16T20:45:45.956640+0000","last_fullsized":"2024-03-16T20:45:45.956640+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:58:24.502319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957020+0000","last_change":"2024-03-16T20:45:44.974971+0000","last_active":"2024-03-16T20:45:45.957020+0000","last_peered":"2024-03-16T20:45:45.957020+0000","last_clean":"2024-03-16T20:45:45.957020+0000","last_became_active":"2024-03-16T20:45:44.973981+0000","last_became_peered":"2024-03-16T20:45:44.973981+0000","last_unstale":"2024-03-16T20:45:45.957020+0000","last_undegraded":"2024-03-16T20:45:45.957020+0000","last_fullsized":"2024-03-16T20:45:45.957020+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:45:12.371596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267024+0000","last_change":"2024-03-16T20:45:43.948526+0000","last_active":"2024-03-16T20:45:46.267024+0000","last_peered":"2024-03-16T20:45:46.267024+0000","last_clean":"2024-03-16T20:45:46.267024+0000","last_became_active":"2024-03-16T20:45:43.948226+0000","last_became_peered":"2024-03-16T20:45:43.948226+0000","last_unstale":"2024-03-16T20:45:46.267024+0000","last_undegraded":"2024-03-16T20:45:46.267024+0000","last_fullsized":"2024-03-16T20:45:46.267024+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:40:03.889251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266987+0000","last_change":"2024-03-16T20:45:44.986140+0000","last_active":"2024-03-16T20:45:46.266987+0000","last_peered":"2024-03-16T20:45:46.266987+0000","last_clean":"2024-03-16T20:45:46.266987+0000","last_became_active":"2024-03-16T20:45:44.985505+0000","last_became_peered":"2024-03-16T20:45:44.985505+0000","last_unstale":"2024-03-16T20:45:46.266987+0000","last_undegraded":"2024-03-16T20:45:46.266987+0000","last_fullsized":"2024-03-16T20:45:46.266987+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:04:57.201475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005726+0000","last_change":"2024-03-16T20:45:43.930479+0000","last_active":"2024-03-16T20:45:46.005726+0000","last_peered":"2024-03-16T20:45:46.005726+0000","last_clean":"2024-03-16T20:45:46.005726+0000","last_became_active":"2024-03-16T20:45:43.929953+0000","last_became_peered":"2024-03-16T20:45:43.929953+0000","last_unstale":"2024-03-16T20:45:46.005726+0000","last_undegraded":"2024-03-16T20:45:46.005726+0000","last_fullsized":"2024-03-16T20:45:46.005726+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:52:52.971592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267140+0000","last_change":"2024-03-16T20:45:44.976577+0000","last_active":"2024-03-16T20:45:46.267140+0000","last_peered":"2024-03-16T20:45:46.267140+0000","last_clean":"2024-03-16T20:45:46.267140+0000","last_became_active":"2024-03-16T20:45:44.975824+0000","last_became_peered":"2024-03-16T20:45:44.975824+0000","last_unstale":"2024-03-16T20:45:46.267140+0000","last_undegraded":"2024-03-16T20:45:46.267140+0000","last_fullsized":"2024-03-16T20:45:46.267140+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:39:41.772078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"29'3","reported_seq":17,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957551+0000","last_change":"2024-03-16T20:45:43.955812+0000","last_active":"2024-03-16T20:45:45.957551+0000","last_peered":"2024-03-16T20:45:45.957551+0000","last_clean":"2024-03-16T20:45:45.957551+0000","last_became_active":"2024-03-16T20:45:43.955052+0000","last_became_peered":"2024-03-16T20:45:43.955052+0000","last_unstale":"2024-03-16T20:45:45.957551+0000","last_undegraded":"2024-03-16T20:45:45.957551+0000","last_fullsized":"2024-03-16T20:45:45.957551+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:29:16.261608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956689+0000","last_change":"2024-03-16T20:45:44.985675+0000","last_active":"2024-03-16T20:45:45.956689+0000","last_peered":"2024-03-16T20:45:45.956689+0000","last_clean":"2024-03-16T20:45:45.956689+0000","last_became_active":"2024-03-16T20:45:44.984881+0000","last_became_peered":"2024-03-16T20:45:44.984881+0000","last_unstale":"2024-03-16T20:45:45.956689+0000","last_undegraded":"2024-03-16T20:45:45.956689+0000","last_fullsized":"2024-03-16T20:45:45.956689+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:06:36.112288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"29'6","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267085+0000","last_change":"2024-03-16T20:45:43.932131+0000","last_active":"2024-03-16T20:45:46.267085+0000","last_peered":"2024-03-16T20:45:46.267085+0000","last_clean":"2024-03-16T20:45:46.267085+0000","last_became_active":"2024-03-16T20:45:43.931603+0000","last_became_peered":"2024-03-16T20:45:43.931603+0000","last_unstale":"2024-03-16T20:45:46.267085+0000","last_undegraded":"2024-03-16T20:45:46.267085+0000","last_fullsized":"2024-03-16T20:45:46.267085+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:25:31.730936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956486+0000","last_change":"2024-03-16T20:45:44.974010+0000","last_active":"2024-03-16T20:45:45.956486+0000","last_peered":"2024-03-16T20:45:45.956486+0000","last_clean":"2024-03-16T20:45:45.956486+0000","last_became_active":"2024-03-16T20:45:44.973236+0000","last_became_peered":"2024-03-16T20:45:44.973236+0000","last_unstale":"2024-03-16T20:45:45.956486+0000","last_undegraded":"2024-03-16T20:45:45.956486+0000","last_fullsized":"2024-03-16T20:45:45.956486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:39:42.030933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.958082+0000","last_change":"2024-03-16T20:45:43.940540+0000","last_active":"2024-03-16T20:45:45.958082+0000","last_peered":"2024-03-16T20:45:45.958082+0000","last_clean":"2024-03-16T20:45:45.958082+0000","last_became_active":"2024-03-16T20:45:43.939463+0000","last_became_peered":"2024-03-16T20:45:43.939463+0000","last_unstale":"2024-03-16T20:45:45.958082+0000","last_undegraded":"2024-03-16T20:45:45.958082+0000","last_fullsized":"2024-03-16T20:45:45.958082+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:46:09.548766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266648+0000","last_change":"2024-03-16T20:45:44.967457+0000","last_active":"2024-03-16T20:45:46.266648+0000","last_peered":"2024-03-16T20:45:46.266648+0000","last_clean":"2024-03-16T20:45:46.266648+0000","last_became_active":"2024-03-16T20:45:44.966729+0000","last_became_peered":"2024-03-16T20:45:44.966729+0000","last_unstale":"2024-03-16T20:45:46.266648+0000","last_undegraded":"2024-03-16T20:45:46.266648+0000","last_fullsized":"2024-03-16T20:45:46.266648+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:20:42.752303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956344+0000","last_change":"2024-03-16T20:45:43.936392+0000","last_active":"2024-03-16T20:45:45.956344+0000","last_peered":"2024-03-16T20:45:45.956344+0000","last_clean":"2024-03-16T20:45:45.956344+0000","last_became_active":"2024-03-16T20:45:43.936050+0000","last_became_peered":"2024-03-16T20:45:43.936050+0000","last_unstale":"2024-03-16T20:45:45.956344+0000","last_undegraded":"2024-03-16T20:45:45.956344+0000","last_fullsized":"2024-03-16T20:45:45.956344+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:29:16.261608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269169+0000","last_change":"2024-03-16T20:45:45.002182+0000","last_active":"2024-03-16T20:45:46.269169+0000","last_peered":"2024-03-16T20:45:46.269169+0000","last_clean":"2024-03-16T20:45:46.269169+0000","last_became_active":"2024-03-16T20:45:45.000789+0000","last_became_peered":"2024-03-16T20:45:45.000789+0000","last_unstale":"2024-03-16T20:45:46.269169+0000","last_undegraded":"2024-03-16T20:45:46.269169+0000","last_fullsized":"2024-03-16T20:45:46.269169+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:02:16.196327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266267+0000","last_change":"2024-03-16T20:45:43.947592+0000","last_active":"2024-03-16T20:45:46.266267+0000","last_peered":"2024-03-16T20:45:46.266267+0000","last_clean":"2024-03-16T20:45:46.266267+0000","last_became_active":"2024-03-16T20:45:43.947229+0000","last_became_peered":"2024-03-16T20:45:43.947229+0000","last_unstale":"2024-03-16T20:45:46.266267+0000","last_undegraded":"2024-03-16T20:45:46.266267+0000","last_fullsized":"2024-03-16T20:45:46.266267+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:41:26.046777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267667+0000","last_change":"2024-03-16T20:45:45.002016+0000","last_active":"2024-03-16T20:45:46.267667+0000","last_peered":"2024-03-16T20:45:46.267667+0000","last_clean":"2024-03-16T20:45:46.267667+0000","last_became_active":"2024-03-16T20:45:45.001286+0000","last_became_peered":"2024-03-16T20:45:45.001286+0000","last_unstale":"2024-03-16T20:45:46.267667+0000","last_undegraded":"2024-03-16T20:45:46.267667+0000","last_fullsized":"2024-03-16T20:45:46.267667+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:07:48.374571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"29'8","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266238+0000","last_change":"2024-03-16T20:45:43.932897+0000","last_active":"2024-03-16T20:45:46.266238+0000","last_peered":"2024-03-16T20:45:46.266238+0000","last_clean":"2024-03-16T20:45:46.266238+0000","last_became_active":"2024-03-16T20:45:43.931665+0000","last_became_peered":"2024-03-16T20:45:43.931665+0000","last_unstale":"2024-03-16T20:45:46.266238+0000","last_undegraded":"2024-03-16T20:45:46.266238+0000","last_fullsized":"2024-03-16T20:45:46.266238+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:45:22.470578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956126+0000","last_change":"2024-03-16T20:45:44.999741+0000","last_active":"2024-03-16T20:45:45.956126+0000","last_peered":"2024-03-16T20:45:45.956126+0000","last_clean":"2024-03-16T20:45:45.956126+0000","last_became_active":"2024-03-16T20:45:44.998998+0000","last_became_peered":"2024-03-16T20:45:44.998998+0000","last_unstale":"2024-03-16T20:45:45.956126+0000","last_undegraded":"2024-03-16T20:45:45.956126+0000","last_fullsized":"2024-03-16T20:45:45.956126+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:07:48.374571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.958152+0000","last_change":"2024-03-16T20:45:43.932373+0000","last_active":"2024-03-16T20:45:45.958152+0000","last_peered":"2024-03-16T20:45:45.958152+0000","last_clean":"2024-03-16T20:45:45.958152+0000","last_became_active":"2024-03-16T20:45:43.931593+0000","last_became_peered":"2024-03-16T20:45:43.931593+0000","last_unstale":"2024-03-16T20:45:45.958152+0000","last_undegraded":"2024-03-16T20:45:45.958152+0000","last_fullsized":"2024-03-16T20:45:45.958152+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:29:54.412142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957975+0000","last_change":"2024-03-16T20:45:45.003331+0000","last_active":"2024-03-16T20:45:45.957975+0000","last_peered":"2024-03-16T20:45:45.957975+0000","last_clean":"2024-03-16T20:45:45.957975+0000","last_became_active":"2024-03-16T20:45:45.002518+0000","last_became_peered":"2024-03-16T20:45:45.002518+0000","last_unstale":"2024-03-16T20:45:45.957975+0000","last_undegraded":"2024-03-16T20:45:45.957975+0000","last_fullsized":"2024-03-16T20:45:45.957975+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:02:16.196327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956428+0000","last_change":"2024-03-16T20:45:43.942493+0000","last_active":"2024-03-16T20:45:45.956428+0000","last_peered":"2024-03-16T20:45:45.956428+0000","last_clean":"2024-03-16T20:45:45.956428+0000","last_became_active":"2024-03-16T20:45:43.941761+0000","last_became_peered":"2024-03-16T20:45:43.941761+0000","last_unstale":"2024-03-16T20:45:45.956428+0000","last_undegraded":"2024-03-16T20:45:45.956428+0000","last_fullsized":"2024-03-16T20:45:45.956428+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:59:41.199216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957015+0000","last_change":"2024-03-16T20:45:45.002530+0000","last_active":"2024-03-16T20:45:45.957015+0000","last_peered":"2024-03-16T20:45:45.957015+0000","last_clean":"2024-03-16T20:45:45.957015+0000","last_became_active":"2024-03-16T20:45:45.002209+0000","last_became_peered":"2024-03-16T20:45:45.002209+0000","last_unstale":"2024-03-16T20:45:45.957015+0000","last_undegraded":"2024-03-16T20:45:45.957015+0000","last_fullsized":"2024-03-16T20:45:45.957015+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:31:14.070831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956922+0000","last_change":"2024-03-16T20:45:43.928863+0000","last_active":"2024-03-16T20:45:45.956922+0000","last_peered":"2024-03-16T20:45:45.956922+0000","last_clean":"2024-03-16T20:45:45.956922+0000","last_became_active":"2024-03-16T20:45:43.928073+0000","last_became_peered":"2024-03-16T20:45:43.928073+0000","last_unstale":"2024-03-16T20:45:45.956922+0000","last_undegraded":"2024-03-16T20:45:45.956922+0000","last_fullsized":"2024-03-16T20:45:45.956922+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:54:33.705576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267925+0000","last_change":"2024-03-16T20:45:44.976132+0000","last_active":"2024-03-16T20:45:46.267925+0000","last_peered":"2024-03-16T20:45:46.267925+0000","last_clean":"2024-03-16T20:45:46.267925+0000","last_became_active":"2024-03-16T20:45:44.975425+0000","last_became_peered":"2024-03-16T20:45:44.975425+0000","last_unstale":"2024-03-16T20:45:46.267925+0000","last_undegraded":"2024-03-16T20:45:46.267925+0000","last_fullsized":"2024-03-16T20:45:46.267925+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:04:10.463024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957127+0000","last_change":"2024-03-16T20:45:43.933789+0000","last_active":"2024-03-16T20:45:45.957127+0000","last_peered":"2024-03-16T20:45:45.957127+0000","last_clean":"2024-03-16T20:45:45.957127+0000","last_became_active":"2024-03-16T20:45:43.933013+0000","last_became_peered":"2024-03-16T20:45:43.933013+0000","last_unstale":"2024-03-16T20:45:45.957127+0000","last_undegraded":"2024-03-16T20:45:45.957127+0000","last_fullsized":"2024-03-16T20:45:45.957127+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:42:09.843074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267621+0000","last_change":"2024-03-16T20:45:45.000163+0000","last_active":"2024-03-16T20:45:46.267621+0000","last_peered":"2024-03-16T20:45:46.267621+0000","last_clean":"2024-03-16T20:45:46.267621+0000","last_became_active":"2024-03-16T20:45:44.999781+0000","last_became_peered":"2024-03-16T20:45:44.999781+0000","last_unstale":"2024-03-16T20:45:46.267621+0000","last_undegraded":"2024-03-16T20:45:46.267621+0000","last_fullsized":"2024-03-16T20:45:46.267621+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:58:25.510895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005354+0000","last_change":"2024-03-16T20:45:43.934020+0000","last_active":"2024-03-16T20:45:46.005354+0000","last_peered":"2024-03-16T20:45:46.005354+0000","last_clean":"2024-03-16T20:45:46.005354+0000","last_became_active":"2024-03-16T20:45:43.933382+0000","last_became_peered":"2024-03-16T20:45:43.933382+0000","last_unstale":"2024-03-16T20:45:46.005354+0000","last_undegraded":"2024-03-16T20:45:46.005354+0000","last_fullsized":"2024-03-16T20:45:46.005354+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:07:33.678359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267811+0000","last_change":"2024-03-16T20:45:44.974563+0000","last_active":"2024-03-16T20:45:46.267811+0000","last_peered":"2024-03-16T20:45:46.267811+0000","last_clean":"2024-03-16T20:45:46.267811+0000","last_became_active":"2024-03-16T20:45:44.968061+0000","last_became_peered":"2024-03-16T20:45:44.968061+0000","last_unstale":"2024-03-16T20:45:46.267811+0000","last_undegraded":"2024-03-16T20:45:46.267811+0000","last_fullsized":"2024-03-16T20:45:46.267811+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:54:34.714152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268258+0000","last_change":"2024-03-16T20:45:43.938395+0000","last_active":"2024-03-16T20:45:46.268258+0000","last_peered":"2024-03-16T20:45:46.268258+0000","last_clean":"2024-03-16T20:45:46.268258+0000","last_became_active":"2024-03-16T20:45:43.937726+0000","last_became_peered":"2024-03-16T20:45:43.937726+0000","last_unstale":"2024-03-16T20:45:46.268258+0000","last_undegraded":"2024-03-16T20:45:46.268258+0000","last_fullsized":"2024-03-16T20:45:46.268258+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:07:33.678359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267499+0000","last_change":"2024-03-16T20:45:44.976550+0000","last_active":"2024-03-16T20:45:46.267499+0000","last_peered":"2024-03-16T20:45:46.267499+0000","last_clean":"2024-03-16T20:45:46.267499+0000","last_became_active":"2024-03-16T20:45:44.975758+0000","last_became_peered":"2024-03-16T20:45:44.975758+0000","last_unstale":"2024-03-16T20:45:46.267499+0000","last_undegraded":"2024-03-16T20:45:46.267499+0000","last_fullsized":"2024-03-16T20:45:46.267499+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:19:27.805219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956027+0000","last_change":"2024-03-16T20:45:43.940698+0000","last_active":"2024-03-16T20:45:45.956027+0000","last_peered":"2024-03-16T20:45:45.956027+0000","last_clean":"2024-03-16T20:45:45.956027+0000","last_became_active":"2024-03-16T20:45:43.940118+0000","last_became_peered":"2024-03-16T20:45:43.940118+0000","last_unstale":"2024-03-16T20:45:45.956027+0000","last_undegraded":"2024-03-16T20:45:45.956027+0000","last_fullsized":"2024-03-16T20:45:45.956027+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:19:26.796643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269086+0000","last_change":"2024-03-16T20:45:44.974628+0000","last_active":"2024-03-16T20:45:46.269086+0000","last_peered":"2024-03-16T20:45:46.269086+0000","last_clean":"2024-03-16T20:45:46.269086+0000","last_became_active":"2024-03-16T20:45:44.973407+0000","last_became_peered":"2024-03-16T20:45:44.973407+0000","last_unstale":"2024-03-16T20:45:46.269086+0000","last_undegraded":"2024-03-16T20:45:46.269086+0000","last_fullsized":"2024-03-16T20:45:46.269086+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:59:19.229572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.955147+0000","last_change":"2024-03-16T20:45:43.952550+0000","last_active":"2024-03-16T20:45:45.955147+0000","last_peered":"2024-03-16T20:45:45.955147+0000","last_clean":"2024-03-16T20:45:45.955147+0000","last_became_active":"2024-03-16T20:45:43.951942+0000","last_became_peered":"2024-03-16T20:45:43.951942+0000","last_unstale":"2024-03-16T20:45:45.955147+0000","last_undegraded":"2024-03-16T20:45:45.955147+0000","last_fullsized":"2024-03-16T20:45:45.955147+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:52:52.971592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267666+0000","last_change":"2024-03-16T20:45:44.979710+0000","last_active":"2024-03-16T20:45:46.267666+0000","last_peered":"2024-03-16T20:45:46.267666+0000","last_clean":"2024-03-16T20:45:46.267666+0000","last_became_active":"2024-03-16T20:45:44.976874+0000","last_became_peered":"2024-03-16T20:45:44.976874+0000","last_unstale":"2024-03-16T20:45:46.267666+0000","last_undegraded":"2024-03-16T20:45:46.267666+0000","last_fullsized":"2024-03-16T20:45:46.267666+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:05:21.873542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.958317+0000","last_change":"2024-03-16T20:45:43.955309+0000","last_active":"2024-03-16T20:45:45.958317+0000","last_peered":"2024-03-16T20:45:45.958317+0000","last_clean":"2024-03-16T20:45:45.958317+0000","last_became_active":"2024-03-16T20:45:43.954978+0000","last_became_peered":"2024-03-16T20:45:43.954978+0000","last_unstale":"2024-03-16T20:45:45.958317+0000","last_undegraded":"2024-03-16T20:45:45.958317+0000","last_fullsized":"2024-03-16T20:45:45.958317+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:11:09.879241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957959+0000","last_change":"2024-03-16T20:45:44.984438+0000","last_active":"2024-03-16T20:45:45.957959+0000","last_peered":"2024-03-16T20:45:45.957959+0000","last_clean":"2024-03-16T20:45:45.957959+0000","last_became_active":"2024-03-16T20:45:44.983734+0000","last_became_peered":"2024-03-16T20:45:44.983734+0000","last_unstale":"2024-03-16T20:45:45.957959+0000","last_undegraded":"2024-03-16T20:45:45.957959+0000","last_fullsized":"2024-03-16T20:45:45.957959+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:59:19.229572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"30'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.007829+0000","last_change":"2024-03-16T20:45:43.930228+0000","last_active":"2024-03-16T20:45:46.007829+0000","last_peered":"2024-03-16T20:45:46.007829+0000","last_clean":"2024-03-16T20:45:46.007829+0000","last_became_active":"2024-03-16T20:45:43.929546+0000","last_became_peered":"2024-03-16T20:45:43.929546+0000","last_unstale":"2024-03-16T20:45:46.007829+0000","last_undegraded":"2024-03-16T20:45:46.007829+0000","last_fullsized":"2024-03-16T20:45:46.007829+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:29:16.261608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005194+0000","last_change":"2024-03-16T20:45:44.973355+0000","last_active":"2024-03-16T20:45:46.005194+0000","last_peered":"2024-03-16T20:45:46.005194+0000","last_clean":"2024-03-16T20:45:46.005194+0000","last_became_active":"2024-03-16T20:45:44.972472+0000","last_became_peered":"2024-03-16T20:45:44.972472+0000","last_unstale":"2024-03-16T20:45:46.005194+0000","last_undegraded":"2024-03-16T20:45:46.005194+0000","last_fullsized":"2024-03-16T20:45:46.005194+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:40:04.897827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957197+0000","last_change":"2024-03-16T20:45:43.927656+0000","last_active":"2024-03-16T20:45:45.957197+0000","last_peered":"2024-03-16T20:45:45.957197+0000","last_clean":"2024-03-16T20:45:45.957197+0000","last_became_active":"2024-03-16T20:45:43.927393+0000","last_became_peered":"2024-03-16T20:45:43.927393+0000","last_unstale":"2024-03-16T20:45:45.957197+0000","last_undegraded":"2024-03-16T20:45:45.957197+0000","last_fullsized":"2024-03-16T20:45:45.957197+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:49:47.476324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267849+0000","last_change":"2024-03-16T20:45:44.986156+0000","last_active":"2024-03-16T20:45:46.267849+0000","last_peered":"2024-03-16T20:45:46.267849+0000","last_clean":"2024-03-16T20:45:46.267849+0000","last_became_active":"2024-03-16T20:45:44.983306+0000","last_became_peered":"2024-03-16T20:45:44.983306+0000","last_unstale":"2024-03-16T20:45:46.267849+0000","last_undegraded":"2024-03-16T20:45:46.267849+0000","last_fullsized":"2024-03-16T20:45:46.267849+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:41:27.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005447+0000","last_change":"2024-03-16T20:45:43.955611+0000","last_active":"2024-03-16T20:45:46.005447+0000","last_peered":"2024-03-16T20:45:46.005447+0000","last_clean":"2024-03-16T20:45:46.005447+0000","last_became_active":"2024-03-16T20:45:43.955171+0000","last_became_peered":"2024-03-16T20:45:43.955171+0000","last_unstale":"2024-03-16T20:45:46.005447+0000","last_undegraded":"2024-03-16T20:45:46.005447+0000","last_fullsized":"2024-03-16T20:45:46.005447+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:29:54.412142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005252+0000","last_change":"2024-03-16T20:45:44.973307+0000","last_active":"2024-03-16T20:45:46.005252+0000","last_peered":"2024-03-16T20:45:46.005252+0000","last_clean":"2024-03-16T20:45:46.005252+0000","last_became_active":"2024-03-16T20:45:44.972319+0000","last_became_peered":"2024-03-16T20:45:44.972319+0000","last_unstale":"2024-03-16T20:45:46.005252+0000","last_undegraded":"2024-03-16T20:45:46.005252+0000","last_fullsized":"2024-03-16T20:45:46.005252+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:57:51.937834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956627+0000","last_change":"2024-03-16T20:45:43.955510+0000","last_active":"2024-03-16T20:45:45.956627+0000","last_peered":"2024-03-16T20:45:45.956627+0000","last_clean":"2024-03-16T20:45:45.956627+0000","last_became_active":"2024-03-16T20:45:43.955111+0000","last_became_peered":"2024-03-16T20:45:43.955111+0000","last_unstale":"2024-03-16T20:45:45.956627+0000","last_undegraded":"2024-03-16T20:45:45.956627+0000","last_fullsized":"2024-03-16T20:45:45.956627+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:52:52.971592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266225+0000","last_change":"2024-03-16T20:45:44.978908+0000","last_active":"2024-03-16T20:45:46.266225+0000","last_peered":"2024-03-16T20:45:46.266225+0000","last_clean":"2024-03-16T20:45:46.266225+0000","last_became_active":"2024-03-16T20:45:44.978200+0000","last_became_peered":"2024-03-16T20:45:44.978200+0000","last_unstale":"2024-03-16T20:45:46.266225+0000","last_undegraded":"2024-03-16T20:45:46.266225+0000","last_fullsized":"2024-03-16T20:45:46.266225+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:02:16.196327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956274+0000","last_change":"2024-03-16T20:45:43.952698+0000","last_active":"2024-03-16T20:45:45.956274+0000","last_peered":"2024-03-16T20:45:45.956274+0000","last_clean":"2024-03-16T20:45:45.956274+0000","last_became_active":"2024-03-16T20:45:43.952125+0000","last_became_peered":"2024-03-16T20:45:43.952125+0000","last_unstale":"2024-03-16T20:45:45.956274+0000","last_undegraded":"2024-03-16T20:45:45.956274+0000","last_fullsized":"2024-03-16T20:45:45.956274+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:40:56.332196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.955744+0000","last_change":"2024-03-16T20:45:44.999202+0000","last_active":"2024-03-16T20:45:45.955744+0000","last_peered":"2024-03-16T20:45:45.955744+0000","last_clean":"2024-03-16T20:45:45.955744+0000","last_became_active":"2024-03-16T20:45:44.997189+0000","last_became_peered":"2024-03-16T20:45:44.997189+0000","last_unstale":"2024-03-16T20:45:45.955744+0000","last_undegraded":"2024-03-16T20:45:45.955744+0000","last_fullsized":"2024-03-16T20:45:45.955744+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:30:32.602019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266347+0000","last_change":"2024-03-16T20:45:43.947648+0000","last_active":"2024-03-16T20:45:46.266347+0000","last_peered":"2024-03-16T20:45:46.266347+0000","last_clean":"2024-03-16T20:45:46.266347+0000","last_became_active":"2024-03-16T20:45:43.947253+0000","last_became_peered":"2024-03-16T20:45:43.947253+0000","last_unstale":"2024-03-16T20:45:46.266347+0000","last_undegraded":"2024-03-16T20:45:46.266347+0000","last_fullsized":"2024-03-16T20:45:46.266347+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:57:50.929258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268421+0000","last_change":"2024-03-16T20:45:44.987174+0000","last_active":"2024-03-16T20:45:46.268421+0000","last_peered":"2024-03-16T20:45:46.268421+0000","last_clean":"2024-03-16T20:45:46.268421+0000","last_became_active":"2024-03-16T20:45:44.986410+0000","last_became_peered":"2024-03-16T20:45:44.986410+0000","last_unstale":"2024-03-16T20:45:46.268421+0000","last_undegraded":"2024-03-16T20:45:46.268421+0000","last_fullsized":"2024-03-16T20:45:46.268421+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:05:57.364887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267821+0000","last_change":"2024-03-16T20:45:43.942843+0000","last_active":"2024-03-16T20:45:46.267821+0000","last_peered":"2024-03-16T20:45:46.267821+0000","last_clean":"2024-03-16T20:45:46.267821+0000","last_became_active":"2024-03-16T20:45:43.942077+0000","last_became_peered":"2024-03-16T20:45:43.942077+0000","last_unstale":"2024-03-16T20:45:46.267821+0000","last_undegraded":"2024-03-16T20:45:46.267821+0000","last_fullsized":"2024-03-16T20:45:46.267821+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:39:14.937436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269499+0000","last_change":"2024-03-16T20:44:54.293418+0000","last_active":"2024-03-16T20:45:46.269499+0000","last_peered":"2024-03-16T20:45:46.269499+0000","last_clean":"2024-03-16T20:45:46.269499+0000","last_became_active":"2024-03-16T20:44:54.291347+0000","last_became_peered":"2024-03-16T20:44:54.291347+0000","last_unstale":"2024-03-16T20:45:46.269499+0000","last_undegraded":"2024-03-16T20:45:46.269499+0000","last_fullsized":"2024-03-16T20:45:46.269499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_clean_scrub_stamp":"2024-03-16T20:44:53.077639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:29:18.318494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":933888,"data_stored":719141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":129299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":129355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":128787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":130317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":128753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":339968,"data_stored":124017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":712751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T20:45:48.949 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-03-16T20:45:49.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41d262b640 1 Processor -- start 2024-03-16T20:45:49.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41d262b640 1 -- start start 2024-03-16T20:45:49.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41d262b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41cc0b52d0 0x7f41cc13e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:49.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41d262b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f41cc0b4820 0x7f41cc070da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:49.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41d262b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41cc144410 con 0x7f41cc0b4ef0 2024-03-16T20:45:49.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41d262b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41cc0b5750 con 0x7f41cc0b4820 2024-03-16T20:45:49.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41d262b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc0b5890 con 0x7f41cc0b52d0 2024-03-16T20:45:49.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41cbfff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f41cc0b4ef0 0x7f41cc13dae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:37418/0 (socket says 172.21.15.122:37418) 2024-03-16T20:45:49.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41cbfff640 1 -- 172.21.15.122:0/161216018 learned_addr learned my addr 172.21.15.122:0/161216018 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:49.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41d0bc8640 1 --2- 172.21.15.122:0/161216018 >> v2:172.21.15.195:3300/0 conn(0x7f41cc0b4820 0x7f41cc070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:49.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41cb7fe640 1 --2- 172.21.15.122:0/161216018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41cc0b52d0 0x7f41cc13e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:49.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41caffd640 1 -- 172.21.15.122:0/161216018 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1469589995 0 0) 0x7f41cc144410 con 0x7f41cc0b4ef0 2024-03-16T20:45:49.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41caffd640 1 -- 172.21.15.122:0/161216018 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41b8003530 con 0x7f41cc0b4ef0 2024-03-16T20:45:49.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41caffd640 1 -- 172.21.15.122:0/161216018 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1578416607 0 0) 0x7f41b8003530 con 0x7f41cc0b4ef0 2024-03-16T20:45:49.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41caffd640 1 -- 172.21.15.122:0/161216018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41cc0b52d0 msgr2=0x7f41cc13e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:49.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41caffd640 1 --2- 172.21.15.122:0/161216018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41cc0b52d0 0x7f41cc13e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41caffd640 1 -- 172.21.15.122:0/161216018 >> v2:172.21.15.195:3300/0 conn(0x7f41cc0b4820 msgr2=0x7f41cc070da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:49.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41caffd640 1 --2- 172.21.15.122:0/161216018 >> v2:172.21.15.195:3300/0 conn(0x7f41cc0b4820 0x7f41cc070da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41caffd640 1 -- 172.21.15.122:0/161216018 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc06eb70 con 0x7f41cc0b4ef0 2024-03-16T20:45:49.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41caffd640 1 -- 172.21.15.122:0/161216018 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f41c4001a00 con 0x7f41cc0b4ef0 2024-03-16T20:45:49.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41caffd640 1 -- 172.21.15.122:0/161216018 >> v1:172.21.15.122:6789/0 conn(0x7f41cc0b4ef0 legacy=0x7f41cc13dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:49.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41caffd640 1 --2- 172.21.15.122:0/161216018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41cc0b52d0 0x7f41b8003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:49.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41caffd640 1 --2- 172.21.15.122:0/161216018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b8003f00 0x7f41b80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:49.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41caffd640 1 --2- 172.21.15.122:0/161216018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41b8004d10 0x7f41b8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:49.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41caffd640 1 -- 172.21.15.122:0/161216018 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc0b5750 con 0x7f41b8004d10 2024-03-16T20:45:49.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41caffd640 1 -- 172.21.15.122:0/161216018 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc0b5890 con 0x7f41b8003f00 2024-03-16T20:45:49.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41caffd640 1 -- 172.21.15.122:0/161216018 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b800b990 con 0x7f41cc0b52d0 2024-03-16T20:45:49.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.103+0000 7f41d0bc8640 1 --2- 172.21.15.122:0/161216018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41b8004d10 0x7f41b8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:49.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.107+0000 7f41cb7fe640 1 --2- 172.21.15.122:0/161216018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b8003f00 0x7f41b80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:49.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.107+0000 7f41cbfff640 1 --2- 172.21.15.122:0/161216018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41cc0b52d0 0x7f41b8003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:49.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.107+0000 7f41d0bc8640 1 -- 172.21.15.122:0/161216018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41cc0b52d0 msgr2=0x7f41b8003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:49.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.107+0000 7f41d0bc8640 1 --2- 172.21.15.122:0/161216018 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41cc0b52d0 0x7f41b8003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.107+0000 7f41d0bc8640 1 -- 172.21.15.122:0/161216018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b8003f00 msgr2=0x7f41b80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:49.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.107+0000 7f41d0bc8640 1 --2- 172.21.15.122:0/161216018 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41b8003f00 0x7f41b80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.107+0000 7f41d0bc8640 1 -- 172.21.15.122:0/161216018 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b8003530 con 0x7f41b8004d10 2024-03-16T20:45:49.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d0bc8640 1 --2- 172.21.15.122:0/161216018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41b8004d10 0x7f41b8005190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f41cc06d1a0 tx=0x7f41b401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d027a08ff2a17fa1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:49.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41caffd640 1 -- 172.21.15.122:0/161216018 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b40a5640 con 0x7f41b8004d10 2024-03-16T20:45:49.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41caffd640 1 -- 172.21.15.122:0/161216018 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b40a57a0 con 0x7f41b8004d10 2024-03-16T20:45:49.114 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41caffd640 1 -- 172.21.15.122:0/161216018 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b40a5b50 con 0x7f41b8004d10 2024-03-16T20:45:49.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d262b640 1 -- 172.21.15.122:0/161216018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41b8004d10 msgr2=0x7f41b8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:49.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d262b640 1 --2- 172.21.15.122:0/161216018 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41b8004d10 0x7f41b8005190 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f41cc06d1a0 tx=0x7f41b401c9f0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d0bc8640 1 -- 172.21.15.122:0/161216018 reap_dead start 2024-03-16T20:45:49.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d262b640 1 -- 172.21.15.122:0/161216018 shutdown_connections 2024-03-16T20:45:49.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d262b640 1 -- 172.21.15.122:0/161216018 >> 172.21.15.122:0/161216018 conn(0x7f41cc06c0a0 msgr2=0x7f41cc06c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:49.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d262b640 1 -- 172.21.15.122:0/161216018 shutdown_connections 2024-03-16T20:45:49.115 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d262b640 1 -- 172.21.15.122:0/161216018 wait complete. 2024-03-16T20:45:49.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d262b640 1 Processor -- start 2024-03-16T20:45:49.116 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d262b640 1 -- start start 2024-03-16T20:45:49.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d262b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41cc06b1f0 0x7f41cc1da1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:49.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d262b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41cc1da730 0x7f41cc0afd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:49.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d262b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41cc0b0250 0x7f41cc1dd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:49.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d262b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc0a9ca0 con 0x7f41cc1da730 2024-03-16T20:45:49.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d262b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc0ab1d0 con 0x7f41cc06b1f0 2024-03-16T20:45:49.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d262b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc06b5d0 con 0x7f41cc0b0250 2024-03-16T20:45:49.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41cb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41cc1da730 0x7f41cc0afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:49.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41cb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41cc1da730 0x7f41cc0afd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49118/0 (socket says 172.21.15.122:49118) 2024-03-16T20:45:49.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41cb7fe640 1 -- 172.21.15.122:0/909335585 learned_addr learned my addr 172.21.15.122:0/909335585 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:49.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41cbfff640 1 --2- 172.21.15.122:0/909335585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41cc06b1f0 0x7f41cc1da1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:49.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41cb7fe640 1 -- 172.21.15.122:0/909335585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41cc0b0250 msgr2=0x7f41cc1dd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:49.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41cb7fe640 1 --2- 172.21.15.122:0/909335585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41cc0b0250 0x7f41cc1dd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41cb7fe640 1 -- 172.21.15.122:0/909335585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41cc06b1f0 msgr2=0x7f41cc1da1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:49.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41cb7fe640 1 --2- 172.21.15.122:0/909335585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41cc06b1f0 0x7f41cc1da1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41cb7fe640 1 -- 172.21.15.122:0/909335585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc0ac4e0 con 0x7f41cc1da730 2024-03-16T20:45:49.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41cbfff640 1 --2- 172.21.15.122:0/909335585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41cc06b1f0 0x7f41cc1da1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:49.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41cb7fe640 1 --2- 172.21.15.122:0/909335585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41cc1da730 0x7f41cc0afd10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f41bc011490 tx=0x7f41bc011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:49.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41c8ff9640 1 -- 172.21.15.122:0/909335585 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc015930 con 0x7f41cc1da730 2024-03-16T20:45:49.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41c8ff9640 1 -- 172.21.15.122:0/909335585 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41bc015a90 con 0x7f41cc1da730 2024-03-16T20:45:49.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d262b640 1 -- 172.21.15.122:0/909335585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41cc0a9fb0 con 0x7f41cc1da730 2024-03-16T20:45:49.120 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41c8ff9640 1 -- 172.21.15.122:0/909335585 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41bc015e40 con 0x7f41cc1da730 2024-03-16T20:45:49.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.111+0000 7f41d262b640 1 -- 172.21.15.122:0/909335585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41cc06b910 con 0x7f41cc1da730 2024-03-16T20:45:49.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.115+0000 7f41d262b640 1 -- 172.21.15.122:0/909335585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41cc0a9e10 con 0x7f41cc1da730 2024-03-16T20:45:49.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.115+0000 7f41c8ff9640 1 -- 172.21.15.122:0/909335585 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f41bc01d400 con 0x7f41cc1da730 2024-03-16T20:45:49.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.115+0000 7f41c8ff9640 1 --2- 172.21.15.122:0/909335585 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f41ac066e70 0x7f41ac069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:49.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.115+0000 7f41c8ff9640 1 -- 172.21.15.122:0/909335585 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41bc0851b0 con 0x7f41cc1da730 2024-03-16T20:45:49.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.115+0000 7f41cbfff640 1 --2- 172.21.15.122:0/909335585 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f41ac066e70 0x7f41ac069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:49.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.115+0000 7f41cbfff640 1 --2- 172.21.15.122:0/909335585 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f41ac066e70 0x7f41ac069330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f41c40013e0 tx=0x7f41c4001370 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:49.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.123+0000 7f41c8ff9640 1 -- 172.21.15.122:0/909335585 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f41cc0a9e10 con 0x7f41cc1da730 2024-03-16T20:45:49.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.335+0000 7f41d262b640 1 -- 172.21.15.122:0/909335585 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f41cc0a9e10 con 0x7f41cc1da730 2024-03-16T20:45:49.340 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.335+0000 7f41c8ff9640 1 -- 172.21.15.122:0/909335585 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2367 (secure 0 0 0) 0x7f41cc0a9e10 con 0x7f41cc1da730 2024-03-16T20:45:49.341 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:49.341 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":69,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":30,"num_osds":8,"num_up_osds":8,"osd_up_since":1710621891,"num_in_osds":8,"osd_in_since":1710621882,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":242343936,"bytes_avail":772851769344,"bytes_total":773094113280,"write_bytes_sec":2522,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":13,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":4995}],"up:standby":1},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-03-16T20:45:46.590426+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-03-16T20:45:49.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.339+0000 7f41d262b640 1 -- 172.21.15.122:0/909335585 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f41ac066e70 msgr2=0x7f41ac069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:49.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.339+0000 7f41d262b640 1 --2- 172.21.15.122:0/909335585 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f41ac066e70 0x7f41ac069330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f41c40013e0 tx=0x7f41c4001370 comp rx=0 tx=0).stop 2024-03-16T20:45:49.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.339+0000 7f41d262b640 1 -- 172.21.15.122:0/909335585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41cc1da730 msgr2=0x7f41cc0afd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:49.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.339+0000 7f41d262b640 1 --2- 172.21.15.122:0/909335585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41cc1da730 0x7f41cc0afd10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f41bc011490 tx=0x7f41bc011950 comp rx=0 tx=0).stop 2024-03-16T20:45:49.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.339+0000 7f41d262b640 1 -- 172.21.15.122:0/909335585 shutdown_connections 2024-03-16T20:45:49.343 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.339+0000 7f41d262b640 1 --2- 172.21.15.122:0/909335585 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f41ac066e70 0x7f41ac069330 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.339+0000 7f41d262b640 1 --2- 172.21.15.122:0/909335585 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f41cc0b0250 0x7f41cc1dd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.339+0000 7f41d262b640 1 --2- 172.21.15.122:0/909335585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f41cc1da730 0x7f41cc0afd10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.339+0000 7f41d262b640 1 --2- 172.21.15.122:0/909335585 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f41cc06b1f0 0x7f41cc1da1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.339+0000 7f41d262b640 1 -- 172.21.15.122:0/909335585 >> 172.21.15.122:0/909335585 conn(0x7f41cc06c0a0 msgr2=0x7f41cc0b75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:49.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.339+0000 7f41d262b640 1 -- 172.21.15.122:0/909335585 shutdown_connections 2024-03-16T20:45:49.344 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.339+0000 7f41d262b640 1 -- 172.21.15.122:0/909335585 wait complete. 2024-03-16T20:45:49.358 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-03-16T20:45:49.358 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T20:45:49.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.467+0000 7f98598ac640 1 Processor -- start 2024-03-16T20:45:49.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.467+0000 7f98598ac640 1 -- start start 2024-03-16T20:45:49.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.471+0000 7f98598ac640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98540b3110 0x7f985413ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:49.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.471+0000 7f98598ac640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f985413f180 0x7f9854148650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:49.475 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.471+0000 7f98598ac640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98540b56f0 con 0x7f9854148b90 2024-03-16T20:45:49.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.471+0000 7f98598ac640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f985406de50 con 0x7f985413f180 2024-03-16T20:45:49.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.471+0000 7f98598ac640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9854149010 con 0x7f98540b3110 2024-03-16T20:45:49.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.471+0000 7f98527fc640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f985413f180 0x7f9854148650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:49.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.471+0000 7f98537fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f9854148b90 0x7f985406af90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:37426/0 (socket says 172.21.15.122:37426) 2024-03-16T20:45:49.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.471+0000 7f98537fe640 1 -- 172.21.15.122:0/736950651 learned_addr learned my addr 172.21.15.122:0/736950651 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:49.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.471+0000 7f9852ffd640 1 --2- 172.21.15.122:0/736950651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98540b3110 0x7f985413ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:49.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.471+0000 7f98527fc640 1 -- 172.21.15.122:0/736950651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98540b3110 msgr2=0x7f985413ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:49.476 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.471+0000 7f98527fc640 1 --2- 172.21.15.122:0/736950651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98540b3110 0x7f985413ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.471+0000 7f98527fc640 1 -- 172.21.15.122:0/736950651 >> v1:172.21.15.122:6789/0 conn(0x7f9854148b90 legacy=0x7f985406af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:49.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.471+0000 7f98527fc640 1 -- 172.21.15.122:0/736950651 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985413f5e0 con 0x7f985413f180 2024-03-16T20:45:49.477 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.471+0000 7f9851ffb640 1 -- 172.21.15.122:0/736950651 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667136646 0 0) 0x7f98540b56f0 con 0x7f9854148b90 2024-03-16T20:45:49.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98527fc640 1 --2- 172.21.15.122:0/736950651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f985413f180 0x7f9854148650 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f98440097c0 tx=0x7f984401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584b6f579fc9989b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:49.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f9851ffb640 1 -- 172.21.15.122:0/736950651 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9844004230 con 0x7f985413f180 2024-03-16T20:45:49.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f9851ffb640 1 -- 172.21.15.122:0/736950651 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98440a7050 con 0x7f985413f180 2024-03-16T20:45:49.479 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f9851ffb640 1 -- 172.21.15.122:0/736950651 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9844003620 con 0x7f985413f180 2024-03-16T20:45:49.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98598ac640 1 -- 172.21.15.122:0/736950651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f985413f180 msgr2=0x7f9854148650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:49.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98598ac640 1 --2- 172.21.15.122:0/736950651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f985413f180 0x7f9854148650 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f98440097c0 tx=0x7f984401c410 comp rx=0 tx=0).stop 2024-03-16T20:45:49.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98598ac640 1 -- 172.21.15.122:0/736950651 shutdown_connections 2024-03-16T20:45:49.480 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98598ac640 1 --2- 172.21.15.122:0/736950651 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f985413f180 0x7f9854148650 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98598ac640 1 --2- 172.21.15.122:0/736950651 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98540b3110 0x7f985413ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98598ac640 1 -- 172.21.15.122:0/736950651 >> 172.21.15.122:0/736950651 conn(0x7f9854070df0 msgr2=0x7f9854071200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:49.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98598ac640 1 -- 172.21.15.122:0/736950651 shutdown_connections 2024-03-16T20:45:49.481 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98598ac640 1 -- 172.21.15.122:0/736950651 wait complete. 2024-03-16T20:45:49.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98598ac640 1 Processor -- start 2024-03-16T20:45:49.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98598ac640 1 -- start start 2024-03-16T20:45:49.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98598ac640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98540b3110 0x7f985414fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:49.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98598ac640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f985413f180 0x7f985414ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:49.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98598ac640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9854148b90 0x7f98541df9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:49.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98598ac640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98541df570 con 0x7f9854148b90 2024-03-16T20:45:49.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98598ac640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98540b2780 con 0x7f985413f180 2024-03-16T20:45:49.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98598ac640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98541430f0 con 0x7f98540b3110 2024-03-16T20:45:49.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98537fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9854148b90 0x7f98541df9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:49.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.475+0000 7f98537fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9854148b90 0x7f98541df9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49138/0 (socket says 172.21.15.122:49138) 2024-03-16T20:45:49.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f98537fe640 1 -- 172.21.15.122:0/3343766376 learned_addr learned my addr 172.21.15.122:0/3343766376 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:49.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f98527fc640 1 --2- 172.21.15.122:0/3343766376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f985413f180 0x7f985414ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:49.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f98537fe640 1 -- 172.21.15.122:0/3343766376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98540b3110 msgr2=0x7f985414fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:49.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f98537fe640 1 --2- 172.21.15.122:0/3343766376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98540b3110 0x7f985414fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f98537fe640 1 -- 172.21.15.122:0/3343766376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f985413f180 msgr2=0x7f985414ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:49.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f98537fe640 1 --2- 172.21.15.122:0/3343766376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f985413f180 0x7f985414ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f98537fe640 1 -- 172.21.15.122:0/3343766376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9844009420 con 0x7f9854148b90 2024-03-16T20:45:49.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f98527fc640 1 --2- 172.21.15.122:0/3343766376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f985413f180 0x7f985414ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:49.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f98537fe640 1 --2- 172.21.15.122:0/3343766376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9854148b90 0x7f98541df9c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c700 tx=0x7f984c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:49.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f983bfff640 1 -- 172.21.15.122:0/3343766376 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c00ce10 con 0x7f9854148b90 2024-03-16T20:45:49.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f983bfff640 1 -- 172.21.15.122:0/3343766376 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984c00a640 con 0x7f9854148b90 2024-03-16T20:45:49.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f98598ac640 1 -- 172.21.15.122:0/3343766376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98541433d0 con 0x7f9854148b90 2024-03-16T20:45:49.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f983bfff640 1 -- 172.21.15.122:0/3343766376 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984c00aa70 con 0x7f9854148b90 2024-03-16T20:45:49.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f98598ac640 1 -- 172.21.15.122:0/3343766376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9854142d70 con 0x7f9854148b90 2024-03-16T20:45:49.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f98598ac640 1 -- 172.21.15.122:0/3343766376 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985414f6c0 con 0x7f9854148b90 2024-03-16T20:45:49.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f983bfff640 1 -- 172.21.15.122:0/3343766376 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f984c00abd0 con 0x7f9854148b90 2024-03-16T20:45:49.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f983bfff640 1 --2- 172.21.15.122:0/3343766376 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9840066e40 0x7f9840069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:49.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f983bfff640 1 -- 172.21.15.122:0/3343766376 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f984c084120 con 0x7f9854148b90 2024-03-16T20:45:49.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f9852ffd640 1 --2- 172.21.15.122:0/3343766376 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9840066e40 0x7f9840069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:49.487 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.479+0000 7f9852ffd640 1 --2- 172.21.15.122:0/3343766376 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9840066e40 0x7f9840069300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f985413ba00 tx=0x7f9848008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:49.489 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.483+0000 7f983bfff640 1 -- 172.21.15.122:0/3343766376 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f984c050fa0 con 0x7f9854148b90 2024-03-16T20:45:49.632 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.627+0000 7f98598ac640 1 -- 172.21.15.122:0/3343766376 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98540b5380 con 0x7f9840066e40 2024-03-16T20:45:49.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.643+0000 7f983bfff640 1 -- 172.21.15.122:0/3343766376 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251945 (secure 0 0 0) 0x7f984800a4a0 con 0x7f9840066e40 2024-03-16T20:45:49.652 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:49.654 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-03-16T20:45:49.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.647+0000 7f98598ac640 1 -- 172.21.15.122:0/3343766376 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9840066e40 msgr2=0x7f9840069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:49.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.647+0000 7f98598ac640 1 --2- 172.21.15.122:0/3343766376 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9840066e40 0x7f9840069300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f985413ba00 tx=0x7f9848008040 comp rx=0 tx=0).stop 2024-03-16T20:45:49.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.647+0000 7f98598ac640 1 -- 172.21.15.122:0/3343766376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9854148b90 msgr2=0x7f98541df9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:49.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.647+0000 7f98598ac640 1 --2- 172.21.15.122:0/3343766376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9854148b90 0x7f98541df9c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c700 tx=0x7f984c00cbc0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.647+0000 7f98598ac640 1 -- 172.21.15.122:0/3343766376 shutdown_connections 2024-03-16T20:45:49.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.647+0000 7f98598ac640 1 --2- 172.21.15.122:0/3343766376 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9840066e40 0x7f9840069300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.647+0000 7f98598ac640 1 --2- 172.21.15.122:0/3343766376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9854148b90 0x7f98541df9c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.647+0000 7f98598ac640 1 --2- 172.21.15.122:0/3343766376 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f985413f180 0x7f985414ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.647+0000 7f98598ac640 1 --2- 172.21.15.122:0/3343766376 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f98540b3110 0x7f985414fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:49.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.647+0000 7f98598ac640 1 -- 172.21.15.122:0/3343766376 >> 172.21.15.122:0/3343766376 conn(0x7f9854070df0 msgr2=0x7f9854074320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:49.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.647+0000 7f98598ac640 1 -- 172.21.15.122:0/3343766376 shutdown_connections 2024-03-16T20:45:49.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:49.647+0000 7f98598ac640 1 -- 172.21.15.122:0/3343766376 wait complete. 2024-03-16T20:45:49.669 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-03-16T20:45:48.591007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236664,"kb_used_data":3824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738056,"statfs":{"total":773094113280,"available":772851769344,"internally_reserved":0,"allocated":3915776,"data_stored":2202420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.682889"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268260+0000","last_change":"2024-03-16T20:45:44.977479+0000","last_active":"2024-03-16T20:45:46.268260+0000","last_peered":"2024-03-16T20:45:46.268260+0000","last_clean":"2024-03-16T20:45:46.268260+0000","last_became_active":"2024-03-16T20:45:44.974377+0000","last_became_peered":"2024-03-16T20:45:44.974377+0000","last_unstale":"2024-03-16T20:45:46.268260+0000","last_undegraded":"2024-03-16T20:45:46.268260+0000","last_fullsized":"2024-03-16T20:45:46.268260+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:40:04.897827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266582+0000","last_change":"2024-03-16T20:45:44.969210+0000","last_active":"2024-03-16T20:45:46.266582+0000","last_peered":"2024-03-16T20:45:46.266582+0000","last_clean":"2024-03-16T20:45:46.266582+0000","last_became_active":"2024-03-16T20:45:44.968476+0000","last_became_peered":"2024-03-16T20:45:44.968476+0000","last_unstale":"2024-03-16T20:45:46.266582+0000","last_undegraded":"2024-03-16T20:45:46.266582+0000","last_fullsized":"2024-03-16T20:45:46.266582+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:54:20.920066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957286+0000","last_change":"2024-03-16T20:45:44.986911+0000","last_active":"2024-03-16T20:45:45.957286+0000","last_peered":"2024-03-16T20:45:45.957286+0000","last_clean":"2024-03-16T20:45:45.957286+0000","last_became_active":"2024-03-16T20:45:44.984194+0000","last_became_peered":"2024-03-16T20:45:44.984194+0000","last_unstale":"2024-03-16T20:45:45.957286+0000","last_undegraded":"2024-03-16T20:45:45.957286+0000","last_fullsized":"2024-03-16T20:45:45.957286+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:49.546012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957075+0000","last_change":"2024-03-16T20:45:45.002983+0000","last_active":"2024-03-16T20:45:45.957075+0000","last_peered":"2024-03-16T20:45:45.957075+0000","last_clean":"2024-03-16T20:45:45.957075+0000","last_became_active":"2024-03-16T20:45:45.002338+0000","last_became_peered":"2024-03-16T20:45:45.002338+0000","last_unstale":"2024-03-16T20:45:45.957075+0000","last_undegraded":"2024-03-16T20:45:45.957075+0000","last_fullsized":"2024-03-16T20:45:45.957075+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:05:57.364887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267523+0000","last_change":"2024-03-16T20:45:44.980896+0000","last_active":"2024-03-16T20:45:46.267523+0000","last_peered":"2024-03-16T20:45:46.267523+0000","last_clean":"2024-03-16T20:45:46.267523+0000","last_became_active":"2024-03-16T20:45:44.979594+0000","last_became_peered":"2024-03-16T20:45:44.979594+0000","last_unstale":"2024-03-16T20:45:46.267523+0000","last_undegraded":"2024-03-16T20:45:46.267523+0000","last_fullsized":"2024-03-16T20:45:46.267523+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:40:57.340772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266581+0000","last_change":"2024-03-16T20:45:44.975175+0000","last_active":"2024-03-16T20:45:46.266581+0000","last_peered":"2024-03-16T20:45:46.266581+0000","last_clean":"2024-03-16T20:45:46.266581+0000","last_became_active":"2024-03-16T20:45:44.974424+0000","last_became_peered":"2024-03-16T20:45:44.974424+0000","last_unstale":"2024-03-16T20:45:46.266581+0000","last_undegraded":"2024-03-16T20:45:46.266581+0000","last_fullsized":"2024-03-16T20:45:46.266581+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:08:10.036696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269399+0000","last_change":"2024-03-16T20:45:44.979934+0000","last_active":"2024-03-16T20:45:46.269399+0000","last_peered":"2024-03-16T20:45:46.269399+0000","last_clean":"2024-03-16T20:45:46.269399+0000","last_became_active":"2024-03-16T20:45:44.979060+0000","last_became_peered":"2024-03-16T20:45:44.979060+0000","last_unstale":"2024-03-16T20:45:46.269399+0000","last_undegraded":"2024-03-16T20:45:46.269399+0000","last_fullsized":"2024-03-16T20:45:46.269399+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:54:20.920066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268330+0000","last_change":"2024-03-16T20:45:44.987586+0000","last_active":"2024-03-16T20:45:46.268330+0000","last_peered":"2024-03-16T20:45:46.268330+0000","last_clean":"2024-03-16T20:45:46.268330+0000","last_became_active":"2024-03-16T20:45:44.986860+0000","last_became_peered":"2024-03-16T20:45:44.986860+0000","last_unstale":"2024-03-16T20:45:46.268330+0000","last_undegraded":"2024-03-16T20:45:46.268330+0000","last_fullsized":"2024-03-16T20:45:46.268330+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:31:14.070831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956478+0000","last_change":"2024-03-16T20:45:44.997710+0000","last_active":"2024-03-16T20:45:45.956478+0000","last_peered":"2024-03-16T20:45:45.956478+0000","last_clean":"2024-03-16T20:45:45.956478+0000","last_became_active":"2024-03-16T20:45:44.996999+0000","last_became_peered":"2024-03-16T20:45:44.996999+0000","last_unstale":"2024-03-16T20:45:45.956478+0000","last_undegraded":"2024-03-16T20:45:45.956478+0000","last_fullsized":"2024-03-16T20:45:45.956478+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:40:04.897827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957631+0000","last_change":"2024-03-16T20:45:44.971505+0000","last_active":"2024-03-16T20:45:45.957631+0000","last_peered":"2024-03-16T20:45:45.957631+0000","last_clean":"2024-03-16T20:45:45.957631+0000","last_became_active":"2024-03-16T20:45:44.970737+0000","last_became_peered":"2024-03-16T20:45:44.970737+0000","last_unstale":"2024-03-16T20:45:45.957631+0000","last_undegraded":"2024-03-16T20:45:45.957631+0000","last_fullsized":"2024-03-16T20:45:45.957631+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:06:36.112288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269089+0000","last_change":"2024-03-16T20:45:44.988896+0000","last_active":"2024-03-16T20:45:46.269089+0000","last_peered":"2024-03-16T20:45:46.269089+0000","last_clean":"2024-03-16T20:45:46.269089+0000","last_became_active":"2024-03-16T20:45:44.988250+0000","last_became_peered":"2024-03-16T20:45:44.988250+0000","last_unstale":"2024-03-16T20:45:46.269089+0000","last_undegraded":"2024-03-16T20:45:46.269089+0000","last_fullsized":"2024-03-16T20:45:46.269089+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:57:53.975463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956681+0000","last_change":"2024-03-16T20:45:44.969082+0000","last_active":"2024-03-16T20:45:45.956681+0000","last_peered":"2024-03-16T20:45:45.956681+0000","last_clean":"2024-03-16T20:45:45.956681+0000","last_became_active":"2024-03-16T20:45:44.968282+0000","last_became_peered":"2024-03-16T20:45:44.968282+0000","last_unstale":"2024-03-16T20:45:45.956681+0000","last_undegraded":"2024-03-16T20:45:45.956681+0000","last_fullsized":"2024-03-16T20:45:45.956681+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:05:57.364887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267692+0000","last_change":"2024-03-16T20:45:44.984346+0000","last_active":"2024-03-16T20:45:46.267692+0000","last_peered":"2024-03-16T20:45:46.267692+0000","last_clean":"2024-03-16T20:45:46.267692+0000","last_became_active":"2024-03-16T20:45:44.983837+0000","last_became_peered":"2024-03-16T20:45:44.983837+0000","last_unstale":"2024-03-16T20:45:46.267692+0000","last_undegraded":"2024-03-16T20:45:46.267692+0000","last_fullsized":"2024-03-16T20:45:46.267692+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:46:10.557343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266735+0000","last_change":"2024-03-16T20:45:44.969608+0000","last_active":"2024-03-16T20:45:46.266735+0000","last_peered":"2024-03-16T20:45:46.266735+0000","last_clean":"2024-03-16T20:45:46.266735+0000","last_became_active":"2024-03-16T20:45:44.968876+0000","last_became_peered":"2024-03-16T20:45:44.968876+0000","last_unstale":"2024-03-16T20:45:46.266735+0000","last_undegraded":"2024-03-16T20:45:46.266735+0000","last_fullsized":"2024-03-16T20:45:46.266735+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:20:45.687785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.955788+0000","last_change":"2024-03-16T20:45:44.975592+0000","last_active":"2024-03-16T20:45:45.955788+0000","last_peered":"2024-03-16T20:45:45.955788+0000","last_clean":"2024-03-16T20:45:45.955788+0000","last_became_active":"2024-03-16T20:45:44.973733+0000","last_became_peered":"2024-03-16T20:45:44.973733+0000","last_unstale":"2024-03-16T20:45:45.955788+0000","last_undegraded":"2024-03-16T20:45:45.955788+0000","last_fullsized":"2024-03-16T20:45:45.955788+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:31:14.070831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269494+0000","last_change":"2024-03-16T20:45:44.976648+0000","last_active":"2024-03-16T20:45:46.269494+0000","last_peered":"2024-03-16T20:45:46.269494+0000","last_clean":"2024-03-16T20:45:46.269494+0000","last_became_active":"2024-03-16T20:45:44.975885+0000","last_became_peered":"2024-03-16T20:45:44.975885+0000","last_unstale":"2024-03-16T20:45:46.269494+0000","last_undegraded":"2024-03-16T20:45:46.269494+0000","last_fullsized":"2024-03-16T20:45:46.269494+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:08:10.036696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268613+0000","last_change":"2024-03-16T20:45:44.983529+0000","last_active":"2024-03-16T20:45:46.268613+0000","last_peered":"2024-03-16T20:45:46.268613+0000","last_clean":"2024-03-16T20:45:46.268613+0000","last_became_active":"2024-03-16T20:45:44.982855+0000","last_became_peered":"2024-03-16T20:45:44.982855+0000","last_unstale":"2024-03-16T20:45:46.268613+0000","last_undegraded":"2024-03-16T20:45:46.268613+0000","last_fullsized":"2024-03-16T20:45:46.268613+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:45:12.371596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266734+0000","last_change":"2024-03-16T20:45:44.986663+0000","last_active":"2024-03-16T20:45:46.266734+0000","last_peered":"2024-03-16T20:45:46.266734+0000","last_clean":"2024-03-16T20:45:46.266734+0000","last_became_active":"2024-03-16T20:45:44.985802+0000","last_became_peered":"2024-03-16T20:45:44.985802+0000","last_unstale":"2024-03-16T20:45:46.266734+0000","last_undegraded":"2024-03-16T20:45:46.266734+0000","last_fullsized":"2024-03-16T20:45:46.266734+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:29:17.211554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266824+0000","last_change":"2024-03-16T20:45:44.971503+0000","last_active":"2024-03-16T20:45:46.266824+0000","last_peered":"2024-03-16T20:45:46.266824+0000","last_clean":"2024-03-16T20:45:46.266824+0000","last_became_active":"2024-03-16T20:45:44.970764+0000","last_became_peered":"2024-03-16T20:45:44.970764+0000","last_unstale":"2024-03-16T20:45:46.266824+0000","last_undegraded":"2024-03-16T20:45:46.266824+0000","last_fullsized":"2024-03-16T20:45:46.266824+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:51:07.991442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267798+0000","last_change":"2024-03-16T20:45:44.981615+0000","last_active":"2024-03-16T20:45:46.267798+0000","last_peered":"2024-03-16T20:45:46.267798+0000","last_clean":"2024-03-16T20:45:46.267798+0000","last_became_active":"2024-03-16T20:45:44.980565+0000","last_became_peered":"2024-03-16T20:45:44.980565+0000","last_unstale":"2024-03-16T20:45:46.267798+0000","last_undegraded":"2024-03-16T20:45:46.267798+0000","last_fullsized":"2024-03-16T20:45:46.267798+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:42:10.851650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957994+0000","last_change":"2024-03-16T20:45:45.001667+0000","last_active":"2024-03-16T20:45:45.957994+0000","last_peered":"2024-03-16T20:45:45.957994+0000","last_clean":"2024-03-16T20:45:45.957994+0000","last_became_active":"2024-03-16T20:45:44.999133+0000","last_became_peered":"2024-03-16T20:45:44.999133+0000","last_unstale":"2024-03-16T20:45:45.957994+0000","last_undegraded":"2024-03-16T20:45:45.957994+0000","last_fullsized":"2024-03-16T20:45:45.957994+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:45:23.479155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956254+0000","last_change":"2024-03-16T20:45:44.996905+0000","last_active":"2024-03-16T20:45:45.956254+0000","last_peered":"2024-03-16T20:45:45.956254+0000","last_clean":"2024-03-16T20:45:45.956254+0000","last_became_active":"2024-03-16T20:45:44.996182+0000","last_became_peered":"2024-03-16T20:45:44.996182+0000","last_unstale":"2024-03-16T20:45:45.956254+0000","last_undegraded":"2024-03-16T20:45:45.956254+0000","last_fullsized":"2024-03-16T20:45:45.956254+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:41:27.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005074+0000","last_change":"2024-03-16T20:45:44.972167+0000","last_active":"2024-03-16T20:45:46.005074+0000","last_peered":"2024-03-16T20:45:46.005074+0000","last_clean":"2024-03-16T20:45:46.005074+0000","last_became_active":"2024-03-16T20:45:44.971283+0000","last_became_peered":"2024-03-16T20:45:44.971283+0000","last_unstale":"2024-03-16T20:45:46.005074+0000","last_undegraded":"2024-03-16T20:45:46.005074+0000","last_fullsized":"2024-03-16T20:45:46.005074+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:41:27.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.955247+0000","last_change":"2024-03-16T20:45:44.977536+0000","last_active":"2024-03-16T20:45:45.955247+0000","last_peered":"2024-03-16T20:45:45.955247+0000","last_clean":"2024-03-16T20:45:45.955247+0000","last_became_active":"2024-03-16T20:45:44.976794+0000","last_became_peered":"2024-03-16T20:45:44.976794+0000","last_unstale":"2024-03-16T20:45:45.955247+0000","last_undegraded":"2024-03-16T20:45:45.955247+0000","last_fullsized":"2024-03-16T20:45:45.955247+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:35:59.048311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957835+0000","last_change":"2024-03-16T20:45:44.976909+0000","last_active":"2024-03-16T20:45:45.957835+0000","last_peered":"2024-03-16T20:45:45.957835+0000","last_clean":"2024-03-16T20:45:45.957835+0000","last_became_active":"2024-03-16T20:45:44.976127+0000","last_became_peered":"2024-03-16T20:45:44.976127+0000","last_unstale":"2024-03-16T20:45:45.957835+0000","last_undegraded":"2024-03-16T20:45:45.957835+0000","last_fullsized":"2024-03-16T20:45:45.957835+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:45:12.371596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266763+0000","last_change":"2024-03-16T20:45:44.978761+0000","last_active":"2024-03-16T20:45:46.266763+0000","last_peered":"2024-03-16T20:45:46.266763+0000","last_clean":"2024-03-16T20:45:46.266763+0000","last_became_active":"2024-03-16T20:45:44.974520+0000","last_became_peered":"2024-03-16T20:45:44.974520+0000","last_unstale":"2024-03-16T20:45:46.266763+0000","last_undegraded":"2024-03-16T20:45:46.266763+0000","last_fullsized":"2024-03-16T20:45:46.266763+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:14:14.791268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957901+0000","last_change":"2024-03-16T20:45:45.004121+0000","last_active":"2024-03-16T20:45:45.957901+0000","last_peered":"2024-03-16T20:45:45.957901+0000","last_clean":"2024-03-16T20:45:45.957901+0000","last_became_active":"2024-03-16T20:45:45.003251+0000","last_became_peered":"2024-03-16T20:45:45.003251+0000","last_unstale":"2024-03-16T20:45:45.957901+0000","last_undegraded":"2024-03-16T20:45:45.957901+0000","last_fullsized":"2024-03-16T20:45:45.957901+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:39:41.263058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957018+0000","last_change":"2024-03-16T20:45:44.964295+0000","last_active":"2024-03-16T20:45:45.957018+0000","last_peered":"2024-03-16T20:45:45.957018+0000","last_clean":"2024-03-16T20:45:45.957018+0000","last_became_active":"2024-03-16T20:45:44.963502+0000","last_became_peered":"2024-03-16T20:45:44.963502+0000","last_unstale":"2024-03-16T20:45:45.957018+0000","last_undegraded":"2024-03-16T20:45:45.957018+0000","last_fullsized":"2024-03-16T20:45:45.957018+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:39:41.263058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957545+0000","last_change":"2024-03-16T20:45:45.004296+0000","last_active":"2024-03-16T20:45:45.957545+0000","last_peered":"2024-03-16T20:45:45.957545+0000","last_clean":"2024-03-16T20:45:45.957545+0000","last_became_active":"2024-03-16T20:45:44.997947+0000","last_became_peered":"2024-03-16T20:45:44.997947+0000","last_unstale":"2024-03-16T20:45:45.957545+0000","last_undegraded":"2024-03-16T20:45:45.957545+0000","last_fullsized":"2024-03-16T20:45:45.957545+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:25:32.739512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268596+0000","last_change":"2024-03-16T20:45:44.983629+0000","last_active":"2024-03-16T20:45:46.268596+0000","last_peered":"2024-03-16T20:45:46.268596+0000","last_clean":"2024-03-16T20:45:46.268596+0000","last_became_active":"2024-03-16T20:45:44.979161+0000","last_became_peered":"2024-03-16T20:45:44.979161+0000","last_unstale":"2024-03-16T20:45:46.268596+0000","last_undegraded":"2024-03-16T20:45:46.268596+0000","last_fullsized":"2024-03-16T20:45:46.268596+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:06:36.112288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267550+0000","last_change":"2024-03-16T20:45:44.977815+0000","last_active":"2024-03-16T20:45:46.267550+0000","last_peered":"2024-03-16T20:45:46.267550+0000","last_clean":"2024-03-16T20:45:46.267550+0000","last_became_active":"2024-03-16T20:45:44.975655+0000","last_became_peered":"2024-03-16T20:45:44.975655+0000","last_unstale":"2024-03-16T20:45:46.267550+0000","last_undegraded":"2024-03-16T20:45:46.267550+0000","last_fullsized":"2024-03-16T20:45:46.267550+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:30:32.602019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268765+0000","last_change":"2024-03-16T20:45:45.003806+0000","last_active":"2024-03-16T20:45:46.268765+0000","last_peered":"2024-03-16T20:45:46.268765+0000","last_clean":"2024-03-16T20:45:46.268765+0000","last_became_active":"2024-03-16T20:45:45.002935+0000","last_became_peered":"2024-03-16T20:45:45.002935+0000","last_unstale":"2024-03-16T20:45:46.268765+0000","last_undegraded":"2024-03-16T20:45:46.268765+0000","last_fullsized":"2024-03-16T20:45:46.268765+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:49.546012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005668+0000","last_change":"2024-03-16T20:45:44.977311+0000","last_active":"2024-03-16T20:45:46.005668+0000","last_peered":"2024-03-16T20:45:46.005668+0000","last_clean":"2024-03-16T20:45:46.005668+0000","last_became_active":"2024-03-16T20:45:44.975825+0000","last_became_peered":"2024-03-16T20:45:44.975825+0000","last_unstale":"2024-03-16T20:45:46.005668+0000","last_undegraded":"2024-03-16T20:45:46.005668+0000","last_fullsized":"2024-03-16T20:45:46.005668+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:25:32.739512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"29'4","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269202+0000","last_change":"2024-03-16T20:45:43.956395+0000","last_active":"2024-03-16T20:45:46.269202+0000","last_peered":"2024-03-16T20:45:46.269202+0000","last_clean":"2024-03-16T20:45:46.269202+0000","last_became_active":"2024-03-16T20:45:43.955548+0000","last_became_peered":"2024-03-16T20:45:43.955548+0000","last_unstale":"2024-03-16T20:45:46.269202+0000","last_undegraded":"2024-03-16T20:45:46.269202+0000","last_fullsized":"2024-03-16T20:45:46.269202+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:29:54.412142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268904+0000","last_change":"2024-03-16T20:45:45.001424+0000","last_active":"2024-03-16T20:45:46.268904+0000","last_peered":"2024-03-16T20:45:46.268904+0000","last_clean":"2024-03-16T20:45:46.268904+0000","last_became_active":"2024-03-16T20:45:45.000845+0000","last_became_peered":"2024-03-16T20:45:45.000845+0000","last_unstale":"2024-03-16T20:45:46.268904+0000","last_undegraded":"2024-03-16T20:45:46.268904+0000","last_fullsized":"2024-03-16T20:45:46.268904+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:57:51.937834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005625+0000","last_change":"2024-03-16T20:45:43.941139+0000","last_active":"2024-03-16T20:45:46.005625+0000","last_peered":"2024-03-16T20:45:46.005625+0000","last_clean":"2024-03-16T20:45:46.005625+0000","last_became_active":"2024-03-16T20:45:43.940339+0000","last_became_peered":"2024-03-16T20:45:43.940339+0000","last_unstale":"2024-03-16T20:45:46.005625+0000","last_undegraded":"2024-03-16T20:45:46.005625+0000","last_fullsized":"2024-03-16T20:45:46.005625+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:11:09.879241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956868+0000","last_change":"2024-03-16T20:45:44.999396+0000","last_active":"2024-03-16T20:45:45.956868+0000","last_peered":"2024-03-16T20:45:45.956868+0000","last_clean":"2024-03-16T20:45:45.956868+0000","last_became_active":"2024-03-16T20:45:44.997352+0000","last_became_peered":"2024-03-16T20:45:44.997352+0000","last_unstale":"2024-03-16T20:45:45.956868+0000","last_undegraded":"2024-03-16T20:45:45.956868+0000","last_fullsized":"2024-03-16T20:45:45.956868+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:57:51.937834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266929+0000","last_change":"2024-03-16T20:45:43.932198+0000","last_active":"2024-03-16T20:45:46.266929+0000","last_peered":"2024-03-16T20:45:46.266929+0000","last_clean":"2024-03-16T20:45:46.266929+0000","last_became_active":"2024-03-16T20:45:43.931513+0000","last_became_peered":"2024-03-16T20:45:43.931513+0000","last_unstale":"2024-03-16T20:45:46.266929+0000","last_undegraded":"2024-03-16T20:45:46.266929+0000","last_fullsized":"2024-03-16T20:45:46.266929+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:30:31.593443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957955+0000","last_change":"2024-03-16T20:45:45.003785+0000","last_active":"2024-03-16T20:45:45.957955+0000","last_peered":"2024-03-16T20:45:45.957955+0000","last_clean":"2024-03-16T20:45:45.957955+0000","last_became_active":"2024-03-16T20:45:45.003497+0000","last_became_peered":"2024-03-16T20:45:45.003497+0000","last_unstale":"2024-03-16T20:45:45.957955+0000","last_undegraded":"2024-03-16T20:45:45.957955+0000","last_fullsized":"2024-03-16T20:45:45.957955+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:49.546012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267880+0000","last_change":"2024-03-16T20:45:43.929779+0000","last_active":"2024-03-16T20:45:46.267880+0000","last_peered":"2024-03-16T20:45:46.267880+0000","last_clean":"2024-03-16T20:45:46.267880+0000","last_became_active":"2024-03-16T20:45:43.929496+0000","last_became_peered":"2024-03-16T20:45:43.929496+0000","last_unstale":"2024-03-16T20:45:46.267880+0000","last_undegraded":"2024-03-16T20:45:46.267880+0000","last_fullsized":"2024-03-16T20:45:46.267880+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:35:46.461735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268734+0000","last_change":"2024-03-16T20:45:45.000918+0000","last_active":"2024-03-16T20:45:46.268734+0000","last_peered":"2024-03-16T20:45:46.268734+0000","last_clean":"2024-03-16T20:45:46.268734+0000","last_became_active":"2024-03-16T20:45:45.000298+0000","last_became_peered":"2024-03-16T20:45:45.000298+0000","last_unstale":"2024-03-16T20:45:46.268734+0000","last_undegraded":"2024-03-16T20:45:46.268734+0000","last_fullsized":"2024-03-16T20:45:46.268734+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:39:41.263058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957811+0000","last_change":"2024-03-16T20:45:43.942190+0000","last_active":"2024-03-16T20:45:45.957811+0000","last_peered":"2024-03-16T20:45:45.957811+0000","last_clean":"2024-03-16T20:45:45.957811+0000","last_became_active":"2024-03-16T20:45:43.938832+0000","last_became_peered":"2024-03-16T20:45:43.938832+0000","last_unstale":"2024-03-16T20:45:45.957811+0000","last_undegraded":"2024-03-16T20:45:45.957811+0000","last_fullsized":"2024-03-16T20:45:45.957811+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:04:09.454448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957353+0000","last_change":"2024-03-16T20:45:44.966859+0000","last_active":"2024-03-16T20:45:45.957353+0000","last_peered":"2024-03-16T20:45:45.957353+0000","last_clean":"2024-03-16T20:45:45.957353+0000","last_became_active":"2024-03-16T20:45:44.966087+0000","last_became_peered":"2024-03-16T20:45:44.966087+0000","last_unstale":"2024-03-16T20:45:45.957353+0000","last_undegraded":"2024-03-16T20:45:45.957353+0000","last_fullsized":"2024-03-16T20:45:45.957353+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:57:53.975463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956556+0000","last_change":"2024-03-16T20:45:43.935054+0000","last_active":"2024-03-16T20:45:45.956556+0000","last_peered":"2024-03-16T20:45:45.956556+0000","last_clean":"2024-03-16T20:45:45.956556+0000","last_became_active":"2024-03-16T20:45:43.933153+0000","last_became_peered":"2024-03-16T20:45:43.933153+0000","last_unstale":"2024-03-16T20:45:45.956556+0000","last_undegraded":"2024-03-16T20:45:45.956556+0000","last_fullsized":"2024-03-16T20:45:45.956556+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:05:20.864966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.955670+0000","last_change":"2024-03-16T20:45:44.997096+0000","last_active":"2024-03-16T20:45:45.955670+0000","last_peered":"2024-03-16T20:45:45.955670+0000","last_clean":"2024-03-16T20:45:45.955670+0000","last_became_active":"2024-03-16T20:45:44.996219+0000","last_became_peered":"2024-03-16T20:45:44.996219+0000","last_unstale":"2024-03-16T20:45:45.955670+0000","last_undegraded":"2024-03-16T20:45:45.955670+0000","last_fullsized":"2024-03-16T20:45:45.955670+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:59:40.370844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266166+0000","last_change":"2024-03-16T20:45:43.929969+0000","last_active":"2024-03-16T20:45:46.266166+0000","last_peered":"2024-03-16T20:45:46.266166+0000","last_clean":"2024-03-16T20:45:46.266166+0000","last_became_active":"2024-03-16T20:45:43.929709+0000","last_became_peered":"2024-03-16T20:45:43.929709+0000","last_unstale":"2024-03-16T20:45:46.266166+0000","last_undegraded":"2024-03-16T20:45:46.266166+0000","last_fullsized":"2024-03-16T20:45:46.266166+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:07:47.365995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956485+0000","last_change":"2024-03-16T20:45:44.969933+0000","last_active":"2024-03-16T20:45:45.956485+0000","last_peered":"2024-03-16T20:45:45.956485+0000","last_clean":"2024-03-16T20:45:45.956485+0000","last_became_active":"2024-03-16T20:45:44.969188+0000","last_became_peered":"2024-03-16T20:45:44.969188+0000","last_unstale":"2024-03-16T20:45:45.956485+0000","last_undegraded":"2024-03-16T20:45:45.956485+0000","last_fullsized":"2024-03-16T20:45:45.956485+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:35:59.048311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956640+0000","last_change":"2024-03-16T20:45:43.937169+0000","last_active":"2024-03-16T20:45:45.956640+0000","last_peered":"2024-03-16T20:45:45.956640+0000","last_clean":"2024-03-16T20:45:45.956640+0000","last_became_active":"2024-03-16T20:45:43.936408+0000","last_became_peered":"2024-03-16T20:45:43.936408+0000","last_unstale":"2024-03-16T20:45:45.956640+0000","last_undegraded":"2024-03-16T20:45:45.956640+0000","last_fullsized":"2024-03-16T20:45:45.956640+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:58:24.502319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957020+0000","last_change":"2024-03-16T20:45:44.974971+0000","last_active":"2024-03-16T20:45:45.957020+0000","last_peered":"2024-03-16T20:45:45.957020+0000","last_clean":"2024-03-16T20:45:45.957020+0000","last_became_active":"2024-03-16T20:45:44.973981+0000","last_became_peered":"2024-03-16T20:45:44.973981+0000","last_unstale":"2024-03-16T20:45:45.957020+0000","last_undegraded":"2024-03-16T20:45:45.957020+0000","last_fullsized":"2024-03-16T20:45:45.957020+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:45:12.371596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267024+0000","last_change":"2024-03-16T20:45:43.948526+0000","last_active":"2024-03-16T20:45:46.267024+0000","last_peered":"2024-03-16T20:45:46.267024+0000","last_clean":"2024-03-16T20:45:46.267024+0000","last_became_active":"2024-03-16T20:45:43.948226+0000","last_became_peered":"2024-03-16T20:45:43.948226+0000","last_unstale":"2024-03-16T20:45:46.267024+0000","last_undegraded":"2024-03-16T20:45:46.267024+0000","last_fullsized":"2024-03-16T20:45:46.267024+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:40:03.889251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266987+0000","last_change":"2024-03-16T20:45:44.986140+0000","last_active":"2024-03-16T20:45:46.266987+0000","last_peered":"2024-03-16T20:45:46.266987+0000","last_clean":"2024-03-16T20:45:46.266987+0000","last_became_active":"2024-03-16T20:45:44.985505+0000","last_became_peered":"2024-03-16T20:45:44.985505+0000","last_unstale":"2024-03-16T20:45:46.266987+0000","last_undegraded":"2024-03-16T20:45:46.266987+0000","last_fullsized":"2024-03-16T20:45:46.266987+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:04:57.201475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005726+0000","last_change":"2024-03-16T20:45:43.930479+0000","last_active":"2024-03-16T20:45:46.005726+0000","last_peered":"2024-03-16T20:45:46.005726+0000","last_clean":"2024-03-16T20:45:46.005726+0000","last_became_active":"2024-03-16T20:45:43.929953+0000","last_became_peered":"2024-03-16T20:45:43.929953+0000","last_unstale":"2024-03-16T20:45:46.005726+0000","last_undegraded":"2024-03-16T20:45:46.005726+0000","last_fullsized":"2024-03-16T20:45:46.005726+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:52:52.971592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267140+0000","last_change":"2024-03-16T20:45:44.976577+0000","last_active":"2024-03-16T20:45:46.267140+0000","last_peered":"2024-03-16T20:45:46.267140+0000","last_clean":"2024-03-16T20:45:46.267140+0000","last_became_active":"2024-03-16T20:45:44.975824+0000","last_became_peered":"2024-03-16T20:45:44.975824+0000","last_unstale":"2024-03-16T20:45:46.267140+0000","last_undegraded":"2024-03-16T20:45:46.267140+0000","last_fullsized":"2024-03-16T20:45:46.267140+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:39:41.772078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"29'3","reported_seq":17,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957551+0000","last_change":"2024-03-16T20:45:43.955812+0000","last_active":"2024-03-16T20:45:45.957551+0000","last_peered":"2024-03-16T20:45:45.957551+0000","last_clean":"2024-03-16T20:45:45.957551+0000","last_became_active":"2024-03-16T20:45:43.955052+0000","last_became_peered":"2024-03-16T20:45:43.955052+0000","last_unstale":"2024-03-16T20:45:45.957551+0000","last_undegraded":"2024-03-16T20:45:45.957551+0000","last_fullsized":"2024-03-16T20:45:45.957551+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:29:16.261608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956689+0000","last_change":"2024-03-16T20:45:44.985675+0000","last_active":"2024-03-16T20:45:45.956689+0000","last_peered":"2024-03-16T20:45:45.956689+0000","last_clean":"2024-03-16T20:45:45.956689+0000","last_became_active":"2024-03-16T20:45:44.984881+0000","last_became_peered":"2024-03-16T20:45:44.984881+0000","last_unstale":"2024-03-16T20:45:45.956689+0000","last_undegraded":"2024-03-16T20:45:45.956689+0000","last_fullsized":"2024-03-16T20:45:45.956689+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:06:36.112288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"29'6","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267085+0000","last_change":"2024-03-16T20:45:43.932131+0000","last_active":"2024-03-16T20:45:46.267085+0000","last_peered":"2024-03-16T20:45:46.267085+0000","last_clean":"2024-03-16T20:45:46.267085+0000","last_became_active":"2024-03-16T20:45:43.931603+0000","last_became_peered":"2024-03-16T20:45:43.931603+0000","last_unstale":"2024-03-16T20:45:46.267085+0000","last_undegraded":"2024-03-16T20:45:46.267085+0000","last_fullsized":"2024-03-16T20:45:46.267085+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:25:31.730936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956486+0000","last_change":"2024-03-16T20:45:44.974010+0000","last_active":"2024-03-16T20:45:45.956486+0000","last_peered":"2024-03-16T20:45:45.956486+0000","last_clean":"2024-03-16T20:45:45.956486+0000","last_became_active":"2024-03-16T20:45:44.973236+0000","last_became_peered":"2024-03-16T20:45:44.973236+0000","last_unstale":"2024-03-16T20:45:45.956486+0000","last_undegraded":"2024-03-16T20:45:45.956486+0000","last_fullsized":"2024-03-16T20:45:45.956486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:39:42.030933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.958082+0000","last_change":"2024-03-16T20:45:43.940540+0000","last_active":"2024-03-16T20:45:45.958082+0000","last_peered":"2024-03-16T20:45:45.958082+0000","last_clean":"2024-03-16T20:45:45.958082+0000","last_became_active":"2024-03-16T20:45:43.939463+0000","last_became_peered":"2024-03-16T20:45:43.939463+0000","last_unstale":"2024-03-16T20:45:45.958082+0000","last_undegraded":"2024-03-16T20:45:45.958082+0000","last_fullsized":"2024-03-16T20:45:45.958082+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:46:09.548766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266648+0000","last_change":"2024-03-16T20:45:44.967457+0000","last_active":"2024-03-16T20:45:46.266648+0000","last_peered":"2024-03-16T20:45:46.266648+0000","last_clean":"2024-03-16T20:45:46.266648+0000","last_became_active":"2024-03-16T20:45:44.966729+0000","last_became_peered":"2024-03-16T20:45:44.966729+0000","last_unstale":"2024-03-16T20:45:46.266648+0000","last_undegraded":"2024-03-16T20:45:46.266648+0000","last_fullsized":"2024-03-16T20:45:46.266648+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:20:42.752303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956344+0000","last_change":"2024-03-16T20:45:43.936392+0000","last_active":"2024-03-16T20:45:45.956344+0000","last_peered":"2024-03-16T20:45:45.956344+0000","last_clean":"2024-03-16T20:45:45.956344+0000","last_became_active":"2024-03-16T20:45:43.936050+0000","last_became_peered":"2024-03-16T20:45:43.936050+0000","last_unstale":"2024-03-16T20:45:45.956344+0000","last_undegraded":"2024-03-16T20:45:45.956344+0000","last_fullsized":"2024-03-16T20:45:45.956344+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:29:16.261608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269169+0000","last_change":"2024-03-16T20:45:45.002182+0000","last_active":"2024-03-16T20:45:46.269169+0000","last_peered":"2024-03-16T20:45:46.269169+0000","last_clean":"2024-03-16T20:45:46.269169+0000","last_became_active":"2024-03-16T20:45:45.000789+0000","last_became_peered":"2024-03-16T20:45:45.000789+0000","last_unstale":"2024-03-16T20:45:46.269169+0000","last_undegraded":"2024-03-16T20:45:46.269169+0000","last_fullsized":"2024-03-16T20:45:46.269169+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:02:16.196327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266267+0000","last_change":"2024-03-16T20:45:43.947592+0000","last_active":"2024-03-16T20:45:46.266267+0000","last_peered":"2024-03-16T20:45:46.266267+0000","last_clean":"2024-03-16T20:45:46.266267+0000","last_became_active":"2024-03-16T20:45:43.947229+0000","last_became_peered":"2024-03-16T20:45:43.947229+0000","last_unstale":"2024-03-16T20:45:46.266267+0000","last_undegraded":"2024-03-16T20:45:46.266267+0000","last_fullsized":"2024-03-16T20:45:46.266267+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:41:26.046777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267667+0000","last_change":"2024-03-16T20:45:45.002016+0000","last_active":"2024-03-16T20:45:46.267667+0000","last_peered":"2024-03-16T20:45:46.267667+0000","last_clean":"2024-03-16T20:45:46.267667+0000","last_became_active":"2024-03-16T20:45:45.001286+0000","last_became_peered":"2024-03-16T20:45:45.001286+0000","last_unstale":"2024-03-16T20:45:46.267667+0000","last_undegraded":"2024-03-16T20:45:46.267667+0000","last_fullsized":"2024-03-16T20:45:46.267667+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:07:48.374571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"29'8","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266238+0000","last_change":"2024-03-16T20:45:43.932897+0000","last_active":"2024-03-16T20:45:46.266238+0000","last_peered":"2024-03-16T20:45:46.266238+0000","last_clean":"2024-03-16T20:45:46.266238+0000","last_became_active":"2024-03-16T20:45:43.931665+0000","last_became_peered":"2024-03-16T20:45:43.931665+0000","last_unstale":"2024-03-16T20:45:46.266238+0000","last_undegraded":"2024-03-16T20:45:46.266238+0000","last_fullsized":"2024-03-16T20:45:46.266238+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:45:22.470578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956126+0000","last_change":"2024-03-16T20:45:44.999741+0000","last_active":"2024-03-16T20:45:45.956126+0000","last_peered":"2024-03-16T20:45:45.956126+0000","last_clean":"2024-03-16T20:45:45.956126+0000","last_became_active":"2024-03-16T20:45:44.998998+0000","last_became_peered":"2024-03-16T20:45:44.998998+0000","last_unstale":"2024-03-16T20:45:45.956126+0000","last_undegraded":"2024-03-16T20:45:45.956126+0000","last_fullsized":"2024-03-16T20:45:45.956126+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:07:48.374571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.958152+0000","last_change":"2024-03-16T20:45:43.932373+0000","last_active":"2024-03-16T20:45:45.958152+0000","last_peered":"2024-03-16T20:45:45.958152+0000","last_clean":"2024-03-16T20:45:45.958152+0000","last_became_active":"2024-03-16T20:45:43.931593+0000","last_became_peered":"2024-03-16T20:45:43.931593+0000","last_unstale":"2024-03-16T20:45:45.958152+0000","last_undegraded":"2024-03-16T20:45:45.958152+0000","last_fullsized":"2024-03-16T20:45:45.958152+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:29:54.412142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957975+0000","last_change":"2024-03-16T20:45:45.003331+0000","last_active":"2024-03-16T20:45:45.957975+0000","last_peered":"2024-03-16T20:45:45.957975+0000","last_clean":"2024-03-16T20:45:45.957975+0000","last_became_active":"2024-03-16T20:45:45.002518+0000","last_became_peered":"2024-03-16T20:45:45.002518+0000","last_unstale":"2024-03-16T20:45:45.957975+0000","last_undegraded":"2024-03-16T20:45:45.957975+0000","last_fullsized":"2024-03-16T20:45:45.957975+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:02:16.196327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956428+0000","last_change":"2024-03-16T20:45:43.942493+0000","last_active":"2024-03-16T20:45:45.956428+0000","last_peered":"2024-03-16T20:45:45.956428+0000","last_clean":"2024-03-16T20:45:45.956428+0000","last_became_active":"2024-03-16T20:45:43.941761+0000","last_became_peered":"2024-03-16T20:45:43.941761+0000","last_unstale":"2024-03-16T20:45:45.956428+0000","last_undegraded":"2024-03-16T20:45:45.956428+0000","last_fullsized":"2024-03-16T20:45:45.956428+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:59:41.199216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957015+0000","last_change":"2024-03-16T20:45:45.002530+0000","last_active":"2024-03-16T20:45:45.957015+0000","last_peered":"2024-03-16T20:45:45.957015+0000","last_clean":"2024-03-16T20:45:45.957015+0000","last_became_active":"2024-03-16T20:45:45.002209+0000","last_became_peered":"2024-03-16T20:45:45.002209+0000","last_unstale":"2024-03-16T20:45:45.957015+0000","last_undegraded":"2024-03-16T20:45:45.957015+0000","last_fullsized":"2024-03-16T20:45:45.957015+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:31:14.070831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956922+0000","last_change":"2024-03-16T20:45:43.928863+0000","last_active":"2024-03-16T20:45:45.956922+0000","last_peered":"2024-03-16T20:45:45.956922+0000","last_clean":"2024-03-16T20:45:45.956922+0000","last_became_active":"2024-03-16T20:45:43.928073+0000","last_became_peered":"2024-03-16T20:45:43.928073+0000","last_unstale":"2024-03-16T20:45:45.956922+0000","last_undegraded":"2024-03-16T20:45:45.956922+0000","last_fullsized":"2024-03-16T20:45:45.956922+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:54:33.705576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267925+0000","last_change":"2024-03-16T20:45:44.976132+0000","last_active":"2024-03-16T20:45:46.267925+0000","last_peered":"2024-03-16T20:45:46.267925+0000","last_clean":"2024-03-16T20:45:46.267925+0000","last_became_active":"2024-03-16T20:45:44.975425+0000","last_became_peered":"2024-03-16T20:45:44.975425+0000","last_unstale":"2024-03-16T20:45:46.267925+0000","last_undegraded":"2024-03-16T20:45:46.267925+0000","last_fullsized":"2024-03-16T20:45:46.267925+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:04:10.463024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957127+0000","last_change":"2024-03-16T20:45:43.933789+0000","last_active":"2024-03-16T20:45:45.957127+0000","last_peered":"2024-03-16T20:45:45.957127+0000","last_clean":"2024-03-16T20:45:45.957127+0000","last_became_active":"2024-03-16T20:45:43.933013+0000","last_became_peered":"2024-03-16T20:45:43.933013+0000","last_unstale":"2024-03-16T20:45:45.957127+0000","last_undegraded":"2024-03-16T20:45:45.957127+0000","last_fullsized":"2024-03-16T20:45:45.957127+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:42:09.843074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267621+0000","last_change":"2024-03-16T20:45:45.000163+0000","last_active":"2024-03-16T20:45:46.267621+0000","last_peered":"2024-03-16T20:45:46.267621+0000","last_clean":"2024-03-16T20:45:46.267621+0000","last_became_active":"2024-03-16T20:45:44.999781+0000","last_became_peered":"2024-03-16T20:45:44.999781+0000","last_unstale":"2024-03-16T20:45:46.267621+0000","last_undegraded":"2024-03-16T20:45:46.267621+0000","last_fullsized":"2024-03-16T20:45:46.267621+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:58:25.510895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005354+0000","last_change":"2024-03-16T20:45:43.934020+0000","last_active":"2024-03-16T20:45:46.005354+0000","last_peered":"2024-03-16T20:45:46.005354+0000","last_clean":"2024-03-16T20:45:46.005354+0000","last_became_active":"2024-03-16T20:45:43.933382+0000","last_became_peered":"2024-03-16T20:45:43.933382+0000","last_unstale":"2024-03-16T20:45:46.005354+0000","last_undegraded":"2024-03-16T20:45:46.005354+0000","last_fullsized":"2024-03-16T20:45:46.005354+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:07:33.678359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267811+0000","last_change":"2024-03-16T20:45:44.974563+0000","last_active":"2024-03-16T20:45:46.267811+0000","last_peered":"2024-03-16T20:45:46.267811+0000","last_clean":"2024-03-16T20:45:46.267811+0000","last_became_active":"2024-03-16T20:45:44.968061+0000","last_became_peered":"2024-03-16T20:45:44.968061+0000","last_unstale":"2024-03-16T20:45:46.267811+0000","last_undegraded":"2024-03-16T20:45:46.267811+0000","last_fullsized":"2024-03-16T20:45:46.267811+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:54:34.714152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268258+0000","last_change":"2024-03-16T20:45:43.938395+0000","last_active":"2024-03-16T20:45:46.268258+0000","last_peered":"2024-03-16T20:45:46.268258+0000","last_clean":"2024-03-16T20:45:46.268258+0000","last_became_active":"2024-03-16T20:45:43.937726+0000","last_became_peered":"2024-03-16T20:45:43.937726+0000","last_unstale":"2024-03-16T20:45:46.268258+0000","last_undegraded":"2024-03-16T20:45:46.268258+0000","last_fullsized":"2024-03-16T20:45:46.268258+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:07:33.678359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267499+0000","last_change":"2024-03-16T20:45:44.976550+0000","last_active":"2024-03-16T20:45:46.267499+0000","last_peered":"2024-03-16T20:45:46.267499+0000","last_clean":"2024-03-16T20:45:46.267499+0000","last_became_active":"2024-03-16T20:45:44.975758+0000","last_became_peered":"2024-03-16T20:45:44.975758+0000","last_unstale":"2024-03-16T20:45:46.267499+0000","last_undegraded":"2024-03-16T20:45:46.267499+0000","last_fullsized":"2024-03-16T20:45:46.267499+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:19:27.805219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956027+0000","last_change":"2024-03-16T20:45:43.940698+0000","last_active":"2024-03-16T20:45:45.956027+0000","last_peered":"2024-03-16T20:45:45.956027+0000","last_clean":"2024-03-16T20:45:45.956027+0000","last_became_active":"2024-03-16T20:45:43.940118+0000","last_became_peered":"2024-03-16T20:45:43.940118+0000","last_unstale":"2024-03-16T20:45:45.956027+0000","last_undegraded":"2024-03-16T20:45:45.956027+0000","last_fullsized":"2024-03-16T20:45:45.956027+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:19:26.796643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269086+0000","last_change":"2024-03-16T20:45:44.974628+0000","last_active":"2024-03-16T20:45:46.269086+0000","last_peered":"2024-03-16T20:45:46.269086+0000","last_clean":"2024-03-16T20:45:46.269086+0000","last_became_active":"2024-03-16T20:45:44.973407+0000","last_became_peered":"2024-03-16T20:45:44.973407+0000","last_unstale":"2024-03-16T20:45:46.269086+0000","last_undegraded":"2024-03-16T20:45:46.269086+0000","last_fullsized":"2024-03-16T20:45:46.269086+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:59:19.229572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.955147+0000","last_change":"2024-03-16T20:45:43.952550+0000","last_active":"2024-03-16T20:45:45.955147+0000","last_peered":"2024-03-16T20:45:45.955147+0000","last_clean":"2024-03-16T20:45:45.955147+0000","last_became_active":"2024-03-16T20:45:43.951942+0000","last_became_peered":"2024-03-16T20:45:43.951942+0000","last_unstale":"2024-03-16T20:45:45.955147+0000","last_undegraded":"2024-03-16T20:45:45.955147+0000","last_fullsized":"2024-03-16T20:45:45.955147+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:52:52.971592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267666+0000","last_change":"2024-03-16T20:45:44.979710+0000","last_active":"2024-03-16T20:45:46.267666+0000","last_peered":"2024-03-16T20:45:46.267666+0000","last_clean":"2024-03-16T20:45:46.267666+0000","last_became_active":"2024-03-16T20:45:44.976874+0000","last_became_peered":"2024-03-16T20:45:44.976874+0000","last_unstale":"2024-03-16T20:45:46.267666+0000","last_undegraded":"2024-03-16T20:45:46.267666+0000","last_fullsized":"2024-03-16T20:45:46.267666+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:05:21.873542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.958317+0000","last_change":"2024-03-16T20:45:43.955309+0000","last_active":"2024-03-16T20:45:45.958317+0000","last_peered":"2024-03-16T20:45:45.958317+0000","last_clean":"2024-03-16T20:45:45.958317+0000","last_became_active":"2024-03-16T20:45:43.954978+0000","last_became_peered":"2024-03-16T20:45:43.954978+0000","last_unstale":"2024-03-16T20:45:45.958317+0000","last_undegraded":"2024-03-16T20:45:45.958317+0000","last_fullsized":"2024-03-16T20:45:45.958317+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:11:09.879241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957959+0000","last_change":"2024-03-16T20:45:44.984438+0000","last_active":"2024-03-16T20:45:45.957959+0000","last_peered":"2024-03-16T20:45:45.957959+0000","last_clean":"2024-03-16T20:45:45.957959+0000","last_became_active":"2024-03-16T20:45:44.983734+0000","last_became_peered":"2024-03-16T20:45:44.983734+0000","last_unstale":"2024-03-16T20:45:45.957959+0000","last_undegraded":"2024-03-16T20:45:45.957959+0000","last_fullsized":"2024-03-16T20:45:45.957959+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:59:19.229572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"30'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.007829+0000","last_change":"2024-03-16T20:45:43.930228+0000","last_active":"2024-03-16T20:45:46.007829+0000","last_peered":"2024-03-16T20:45:46.007829+0000","last_clean":"2024-03-16T20:45:46.007829+0000","last_became_active":"2024-03-16T20:45:43.929546+0000","last_became_peered":"2024-03-16T20:45:43.929546+0000","last_unstale":"2024-03-16T20:45:46.007829+0000","last_undegraded":"2024-03-16T20:45:46.007829+0000","last_fullsized":"2024-03-16T20:45:46.007829+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:29:16.261608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005194+0000","last_change":"2024-03-16T20:45:44.973355+0000","last_active":"2024-03-16T20:45:46.005194+0000","last_peered":"2024-03-16T20:45:46.005194+0000","last_clean":"2024-03-16T20:45:46.005194+0000","last_became_active":"2024-03-16T20:45:44.972472+0000","last_became_peered":"2024-03-16T20:45:44.972472+0000","last_unstale":"2024-03-16T20:45:46.005194+0000","last_undegraded":"2024-03-16T20:45:46.005194+0000","last_fullsized":"2024-03-16T20:45:46.005194+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:40:04.897827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957197+0000","last_change":"2024-03-16T20:45:43.927656+0000","last_active":"2024-03-16T20:45:45.957197+0000","last_peered":"2024-03-16T20:45:45.957197+0000","last_clean":"2024-03-16T20:45:45.957197+0000","last_became_active":"2024-03-16T20:45:43.927393+0000","last_became_peered":"2024-03-16T20:45:43.927393+0000","last_unstale":"2024-03-16T20:45:45.957197+0000","last_undegraded":"2024-03-16T20:45:45.957197+0000","last_fullsized":"2024-03-16T20:45:45.957197+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:49:47.476324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267849+0000","last_change":"2024-03-16T20:45:44.986156+0000","last_active":"2024-03-16T20:45:46.267849+0000","last_peered":"2024-03-16T20:45:46.267849+0000","last_clean":"2024-03-16T20:45:46.267849+0000","last_became_active":"2024-03-16T20:45:44.983306+0000","last_became_peered":"2024-03-16T20:45:44.983306+0000","last_unstale":"2024-03-16T20:45:46.267849+0000","last_undegraded":"2024-03-16T20:45:46.267849+0000","last_fullsized":"2024-03-16T20:45:46.267849+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:41:27.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005447+0000","last_change":"2024-03-16T20:45:43.955611+0000","last_active":"2024-03-16T20:45:46.005447+0000","last_peered":"2024-03-16T20:45:46.005447+0000","last_clean":"2024-03-16T20:45:46.005447+0000","last_became_active":"2024-03-16T20:45:43.955171+0000","last_became_peered":"2024-03-16T20:45:43.955171+0000","last_unstale":"2024-03-16T20:45:46.005447+0000","last_undegraded":"2024-03-16T20:45:46.005447+0000","last_fullsized":"2024-03-16T20:45:46.005447+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:29:54.412142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005252+0000","last_change":"2024-03-16T20:45:44.973307+0000","last_active":"2024-03-16T20:45:46.005252+0000","last_peered":"2024-03-16T20:45:46.005252+0000","last_clean":"2024-03-16T20:45:46.005252+0000","last_became_active":"2024-03-16T20:45:44.972319+0000","last_became_peered":"2024-03-16T20:45:44.972319+0000","last_unstale":"2024-03-16T20:45:46.005252+0000","last_undegraded":"2024-03-16T20:45:46.005252+0000","last_fullsized":"2024-03-16T20:45:46.005252+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:57:51.937834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956627+0000","last_change":"2024-03-16T20:45:43.955510+0000","last_active":"2024-03-16T20:45:45.956627+0000","last_peered":"2024-03-16T20:45:45.956627+0000","last_clean":"2024-03-16T20:45:45.956627+0000","last_became_active":"2024-03-16T20:45:43.955111+0000","last_became_peered":"2024-03-16T20:45:43.955111+0000","last_unstale":"2024-03-16T20:45:45.956627+0000","last_undegraded":"2024-03-16T20:45:45.956627+0000","last_fullsized":"2024-03-16T20:45:45.956627+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:52:52.971592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266225+0000","last_change":"2024-03-16T20:45:44.978908+0000","last_active":"2024-03-16T20:45:46.266225+0000","last_peered":"2024-03-16T20:45:46.266225+0000","last_clean":"2024-03-16T20:45:46.266225+0000","last_became_active":"2024-03-16T20:45:44.978200+0000","last_became_peered":"2024-03-16T20:45:44.978200+0000","last_unstale":"2024-03-16T20:45:46.266225+0000","last_undegraded":"2024-03-16T20:45:46.266225+0000","last_fullsized":"2024-03-16T20:45:46.266225+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:02:16.196327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956274+0000","last_change":"2024-03-16T20:45:43.952698+0000","last_active":"2024-03-16T20:45:45.956274+0000","last_peered":"2024-03-16T20:45:45.956274+0000","last_clean":"2024-03-16T20:45:45.956274+0000","last_became_active":"2024-03-16T20:45:43.952125+0000","last_became_peered":"2024-03-16T20:45:43.952125+0000","last_unstale":"2024-03-16T20:45:45.956274+0000","last_undegraded":"2024-03-16T20:45:45.956274+0000","last_fullsized":"2024-03-16T20:45:45.956274+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:40:56.332196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.955744+0000","last_change":"2024-03-16T20:45:44.999202+0000","last_active":"2024-03-16T20:45:45.955744+0000","last_peered":"2024-03-16T20:45:45.955744+0000","last_clean":"2024-03-16T20:45:45.955744+0000","last_became_active":"2024-03-16T20:45:44.997189+0000","last_became_peered":"2024-03-16T20:45:44.997189+0000","last_unstale":"2024-03-16T20:45:45.955744+0000","last_undegraded":"2024-03-16T20:45:45.955744+0000","last_fullsized":"2024-03-16T20:45:45.955744+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:30:32.602019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266347+0000","last_change":"2024-03-16T20:45:43.947648+0000","last_active":"2024-03-16T20:45:46.266347+0000","last_peered":"2024-03-16T20:45:46.266347+0000","last_clean":"2024-03-16T20:45:46.266347+0000","last_became_active":"2024-03-16T20:45:43.947253+0000","last_became_peered":"2024-03-16T20:45:43.947253+0000","last_unstale":"2024-03-16T20:45:46.266347+0000","last_undegraded":"2024-03-16T20:45:46.266347+0000","last_fullsized":"2024-03-16T20:45:46.266347+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:57:50.929258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268421+0000","last_change":"2024-03-16T20:45:44.987174+0000","last_active":"2024-03-16T20:45:46.268421+0000","last_peered":"2024-03-16T20:45:46.268421+0000","last_clean":"2024-03-16T20:45:46.268421+0000","last_became_active":"2024-03-16T20:45:44.986410+0000","last_became_peered":"2024-03-16T20:45:44.986410+0000","last_unstale":"2024-03-16T20:45:46.268421+0000","last_undegraded":"2024-03-16T20:45:46.268421+0000","last_fullsized":"2024-03-16T20:45:46.268421+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:05:57.364887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267821+0000","last_change":"2024-03-16T20:45:43.942843+0000","last_active":"2024-03-16T20:45:46.267821+0000","last_peered":"2024-03-16T20:45:46.267821+0000","last_clean":"2024-03-16T20:45:46.267821+0000","last_became_active":"2024-03-16T20:45:43.942077+0000","last_became_peered":"2024-03-16T20:45:43.942077+0000","last_unstale":"2024-03-16T20:45:46.267821+0000","last_undegraded":"2024-03-16T20:45:46.267821+0000","last_fullsized":"2024-03-16T20:45:46.267821+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:39:14.937436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269499+0000","last_change":"2024-03-16T20:44:54.293418+0000","last_active":"2024-03-16T20:45:46.269499+0000","last_peered":"2024-03-16T20:45:46.269499+0000","last_clean":"2024-03-16T20:45:46.269499+0000","last_became_active":"2024-03-16T20:44:54.291347+0000","last_became_peered":"2024-03-16T20:44:54.291347+0000","last_unstale":"2024-03-16T20:45:46.269499+0000","last_undegraded":"2024-03-16T20:45:46.269499+0000","last_fullsized":"2024-03-16T20:45:46.269499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_clean_scrub_stamp":"2024-03-16T20:44:53.077639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:29:18.318494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":933888,"data_stored":719141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":129299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":129355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":128787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":130317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":128753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":339968,"data_stored":124017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":917504,"data_stored":712751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T20:45:52.675 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T20:45:52.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.799+0000 7fe7a0fbc640 1 Processor -- start 2024-03-16T20:45:52.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.799+0000 7fe7a0fbc640 1 -- start start 2024-03-16T20:45:52.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.799+0000 7fe7a0fbc640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe79c072df0 0x7fe79c09a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:52.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.799+0000 7fe7a0fbc640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79c099d90 con 0x7fe79c066fa0 2024-03-16T20:45:52.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.799+0000 7fe7a0fbc640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79c09bc00 con 0x7fe79c0b5060 2024-03-16T20:45:52.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.799+0000 7fe7a0fbc640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe79c067380 con 0x7fe79c072df0 2024-03-16T20:45:52.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.799+0000 7fe799d9b640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe79c072df0 0x7fe79c09a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:52.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.799+0000 7fe79a59c640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fe79c066fa0 0x7fe79c13ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:37428/0 (socket says 172.21.15.122:37428) 2024-03-16T20:45:52.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.799+0000 7fe79a59c640 1 -- 172.21.15.122:0/45266235 learned_addr learned my addr 172.21.15.122:0/45266235 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:52.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.799+0000 7fe799d9b640 1 -- 172.21.15.122:0/45266235 >> v1:172.21.15.195:6789/0 conn(0x7fe79c0b5060 legacy=0x7fe79c14f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:52.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.799+0000 7fe799d9b640 1 -- 172.21.15.122:0/45266235 >> v1:172.21.15.122:6789/0 conn(0x7fe79c066fa0 legacy=0x7fe79c13ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:52.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.799+0000 7fe799d9b640 1 -- 172.21.15.122:0/45266235 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79c09bea0 con 0x7fe79c072df0 2024-03-16T20:45:52.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.799+0000 7fe79959a640 1 -- 172.21.15.122:0/45266235 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943965815 0 0) 0x7fe79c099d90 con 0x7fe79c066fa0 2024-03-16T20:45:52.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe799d9b640 1 --2- 172.21.15.122:0/45266235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c072df0 0x7fe79c09a220 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe7840097c0 tx=0x7fe78401c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ae0e6f3fb5b13f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:52.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe79959a640 1 -- 172.21.15.122:0/45266235 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7840ab600 con 0x7fe79c072df0 2024-03-16T20:45:52.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe79959a640 1 -- 172.21.15.122:0/45266235 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7840a7050 con 0x7fe79c072df0 2024-03-16T20:45:52.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe79959a640 1 -- 172.21.15.122:0/45266235 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7840ab980 con 0x7fe79c072df0 2024-03-16T20:45:52.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe7a0fbc640 1 -- 172.21.15.122:0/45266235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c072df0 msgr2=0x7fe79c09a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:52.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe7a0fbc640 1 --2- 172.21.15.122:0/45266235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c072df0 0x7fe79c09a220 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe7840097c0 tx=0x7fe78401c340 comp rx=0 tx=0).stop 2024-03-16T20:45:52.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe7a0fbc640 1 -- 172.21.15.122:0/45266235 shutdown_connections 2024-03-16T20:45:52.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe7a0fbc640 1 --2- 172.21.15.122:0/45266235 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c072df0 0x7fe79c09a220 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:52.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe7a0fbc640 1 -- 172.21.15.122:0/45266235 >> 172.21.15.122:0/45266235 conn(0x7fe79c06d320 msgr2=0x7fe79c06d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:52.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe7a0fbc640 1 -- 172.21.15.122:0/45266235 shutdown_connections 2024-03-16T20:45:52.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe7a0fbc640 1 -- 172.21.15.122:0/45266235 wait complete. 2024-03-16T20:45:52.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe7a0fbc640 1 Processor -- start 2024-03-16T20:45:52.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe7a0fbc640 1 -- start start 2024-03-16T20:45:52.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe7a0fbc640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe79c066fa0 0x7fe79c133ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:52.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe7a0fbc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c072df0 0x7fe79c0724f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:52.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe7a0fbc640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe79c0b5060 0x7fe79c137b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:52.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe7a0fbc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c132130 con 0x7fe79c072df0 2024-03-16T20:45:52.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe7a0fbc640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c1349a0 con 0x7fe79c066fa0 2024-03-16T20:45:52.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe7a0fbc640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c134c80 con 0x7fe79c0b5060 2024-03-16T20:45:52.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe799d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c072df0 0x7fe79c0724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:52.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe799d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c072df0 0x7fe79c0724f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49168/0 (socket says 172.21.15.122:49168) 2024-03-16T20:45:52.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe799d9b640 1 -- 172.21.15.122:0/655334397 learned_addr learned my addr 172.21.15.122:0/655334397 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:52.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe79a59c640 1 --2- 172.21.15.122:0/655334397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe79c066fa0 0x7fe79c133ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:52.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe79ad9d640 1 --2- 172.21.15.122:0/655334397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe79c0b5060 0x7fe79c137b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:52.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe799d9b640 1 -- 172.21.15.122:0/655334397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe79c0b5060 msgr2=0x7fe79c137b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:52.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe799d9b640 1 --2- 172.21.15.122:0/655334397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe79c0b5060 0x7fe79c137b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:52.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe799d9b640 1 -- 172.21.15.122:0/655334397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe79c066fa0 msgr2=0x7fe79c133ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:52.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe799d9b640 1 --2- 172.21.15.122:0/655334397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe79c066fa0 0x7fe79c133ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:52.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe799d9b640 1 -- 172.21.15.122:0/655334397 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe784009420 con 0x7fe79c072df0 2024-03-16T20:45:52.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe79ad9d640 1 --2- 172.21.15.122:0/655334397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe79c0b5060 0x7fe79c137b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:52.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe799d9b640 1 --2- 172.21.15.122:0/655334397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c072df0 0x7fe79c0724f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe7840097c0 tx=0x7fe78401ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:52.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.803+0000 7fe792ffd640 1 -- 172.21.15.122:0/655334397 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784004230 con 0x7fe79c072df0 2024-03-16T20:45:52.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.807+0000 7fe792ffd640 1 -- 172.21.15.122:0/655334397 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe784004390 con 0x7fe79c072df0 2024-03-16T20:45:52.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.807+0000 7fe7a0fbc640 1 -- 172.21.15.122:0/655334397 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79c134f60 con 0x7fe79c072df0 2024-03-16T20:45:52.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.807+0000 7fe792ffd640 1 -- 172.21.15.122:0/655334397 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784003840 con 0x7fe79c072df0 2024-03-16T20:45:52.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.807+0000 7fe7a0fbc640 1 -- 172.21.15.122:0/655334397 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79c1380c0 con 0x7fe79c072df0 2024-03-16T20:45:52.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.807+0000 7fe7a0fbc640 1 -- 172.21.15.122:0/655334397 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79c071090 con 0x7fe79c072df0 2024-03-16T20:45:52.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.807+0000 7fe792ffd640 1 -- 172.21.15.122:0/655334397 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe7840039a0 con 0x7fe79c072df0 2024-03-16T20:45:52.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.807+0000 7fe792ffd640 1 --2- 172.21.15.122:0/655334397 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe77c066e70 0x7fe77c069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:52.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.807+0000 7fe792ffd640 1 -- 172.21.15.122:0/655334397 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe78411f2d0 con 0x7fe79c072df0 2024-03-16T20:45:52.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.807+0000 7fe79a59c640 1 --2- 172.21.15.122:0/655334397 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe77c066e70 0x7fe77c069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:52.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.807+0000 7fe79a59c640 1 --2- 172.21.15.122:0/655334397 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe77c066e70 0x7fe77c069330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe78c000f70 tx=0x7fe78c007c10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:52.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.811+0000 7fe792ffd640 1 -- 172.21.15.122:0/655334397 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe79c071090 con 0x7fe79c072df0 2024-03-16T20:45:52.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.967+0000 7fe7a0fbc640 1 -- 172.21.15.122:0/655334397 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe79c071090 con 0x7fe77c066e70 2024-03-16T20:45:52.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.987+0000 7fe792ffd640 1 -- 172.21.15.122:0/655334397 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251839 (secure 0 0 0) 0x7fe79c071090 con 0x7fe77c066e70 2024-03-16T20:45:52.996 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:52.998 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-03-16T20:45:52.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.991+0000 7fe7a0fbc640 1 -- 172.21.15.122:0/655334397 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe77c066e70 msgr2=0x7fe77c069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:52.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.991+0000 7fe7a0fbc640 1 --2- 172.21.15.122:0/655334397 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe77c066e70 0x7fe77c069330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe78c000f70 tx=0x7fe78c007c10 comp rx=0 tx=0).stop 2024-03-16T20:45:52.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.991+0000 7fe7a0fbc640 1 -- 172.21.15.122:0/655334397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c072df0 msgr2=0x7fe79c0724f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:52.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.991+0000 7fe7a0fbc640 1 --2- 172.21.15.122:0/655334397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c072df0 0x7fe79c0724f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe7840097c0 tx=0x7fe78401ab80 comp rx=0 tx=0).stop 2024-03-16T20:45:52.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.991+0000 7fe7a0fbc640 1 -- 172.21.15.122:0/655334397 shutdown_connections 2024-03-16T20:45:52.999 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.991+0000 7fe7a0fbc640 1 --2- 172.21.15.122:0/655334397 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe77c066e70 0x7fe77c069330 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.991+0000 7fe7a0fbc640 1 --2- 172.21.15.122:0/655334397 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe79c0b5060 0x7fe79c137b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.991+0000 7fe7a0fbc640 1 --2- 172.21.15.122:0/655334397 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe79c072df0 0x7fe79c0724f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.991+0000 7fe7a0fbc640 1 --2- 172.21.15.122:0/655334397 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe79c066fa0 0x7fe79c133ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.991+0000 7fe7a0fbc640 1 -- 172.21.15.122:0/655334397 >> 172.21.15.122:0/655334397 conn(0x7fe79c06d320 msgr2=0x7fe79c0b8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:53.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.991+0000 7fe7a0fbc640 1 -- 172.21.15.122:0/655334397 shutdown_connections 2024-03-16T20:45:53.000 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:52.991+0000 7fe7a0fbc640 1 -- 172.21.15.122:0/655334397 wait complete. 2024-03-16T20:45:53.013 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-03-16T20:45:52.592595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236696,"kb_used_data":3848,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738024,"statfs":{"total":773094113280,"available":772851736576,"internally_reserved":0,"allocated":3940352,"data_stored":2215020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002864"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268260+0000","last_change":"2024-03-16T20:45:44.977479+0000","last_active":"2024-03-16T20:45:46.268260+0000","last_peered":"2024-03-16T20:45:46.268260+0000","last_clean":"2024-03-16T20:45:46.268260+0000","last_became_active":"2024-03-16T20:45:44.974377+0000","last_became_peered":"2024-03-16T20:45:44.974377+0000","last_unstale":"2024-03-16T20:45:46.268260+0000","last_undegraded":"2024-03-16T20:45:46.268260+0000","last_fullsized":"2024-03-16T20:45:46.268260+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:40:04.897827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266582+0000","last_change":"2024-03-16T20:45:44.969210+0000","last_active":"2024-03-16T20:45:46.266582+0000","last_peered":"2024-03-16T20:45:46.266582+0000","last_clean":"2024-03-16T20:45:46.266582+0000","last_became_active":"2024-03-16T20:45:44.968476+0000","last_became_peered":"2024-03-16T20:45:44.968476+0000","last_unstale":"2024-03-16T20:45:46.266582+0000","last_undegraded":"2024-03-16T20:45:46.266582+0000","last_fullsized":"2024-03-16T20:45:46.266582+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:54:20.920066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957286+0000","last_change":"2024-03-16T20:45:44.986911+0000","last_active":"2024-03-16T20:45:45.957286+0000","last_peered":"2024-03-16T20:45:45.957286+0000","last_clean":"2024-03-16T20:45:45.957286+0000","last_became_active":"2024-03-16T20:45:44.984194+0000","last_became_peered":"2024-03-16T20:45:44.984194+0000","last_unstale":"2024-03-16T20:45:45.957286+0000","last_undegraded":"2024-03-16T20:45:45.957286+0000","last_fullsized":"2024-03-16T20:45:45.957286+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:49.546012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957075+0000","last_change":"2024-03-16T20:45:45.002983+0000","last_active":"2024-03-16T20:45:45.957075+0000","last_peered":"2024-03-16T20:45:45.957075+0000","last_clean":"2024-03-16T20:45:45.957075+0000","last_became_active":"2024-03-16T20:45:45.002338+0000","last_became_peered":"2024-03-16T20:45:45.002338+0000","last_unstale":"2024-03-16T20:45:45.957075+0000","last_undegraded":"2024-03-16T20:45:45.957075+0000","last_fullsized":"2024-03-16T20:45:45.957075+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:05:57.364887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267523+0000","last_change":"2024-03-16T20:45:44.980896+0000","last_active":"2024-03-16T20:45:46.267523+0000","last_peered":"2024-03-16T20:45:46.267523+0000","last_clean":"2024-03-16T20:45:46.267523+0000","last_became_active":"2024-03-16T20:45:44.979594+0000","last_became_peered":"2024-03-16T20:45:44.979594+0000","last_unstale":"2024-03-16T20:45:46.267523+0000","last_undegraded":"2024-03-16T20:45:46.267523+0000","last_fullsized":"2024-03-16T20:45:46.267523+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:40:57.340772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266581+0000","last_change":"2024-03-16T20:45:44.975175+0000","last_active":"2024-03-16T20:45:46.266581+0000","last_peered":"2024-03-16T20:45:46.266581+0000","last_clean":"2024-03-16T20:45:46.266581+0000","last_became_active":"2024-03-16T20:45:44.974424+0000","last_became_peered":"2024-03-16T20:45:44.974424+0000","last_unstale":"2024-03-16T20:45:46.266581+0000","last_undegraded":"2024-03-16T20:45:46.266581+0000","last_fullsized":"2024-03-16T20:45:46.266581+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:08:10.036696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269399+0000","last_change":"2024-03-16T20:45:44.979934+0000","last_active":"2024-03-16T20:45:46.269399+0000","last_peered":"2024-03-16T20:45:46.269399+0000","last_clean":"2024-03-16T20:45:46.269399+0000","last_became_active":"2024-03-16T20:45:44.979060+0000","last_became_peered":"2024-03-16T20:45:44.979060+0000","last_unstale":"2024-03-16T20:45:46.269399+0000","last_undegraded":"2024-03-16T20:45:46.269399+0000","last_fullsized":"2024-03-16T20:45:46.269399+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:54:20.920066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268330+0000","last_change":"2024-03-16T20:45:44.987586+0000","last_active":"2024-03-16T20:45:46.268330+0000","last_peered":"2024-03-16T20:45:46.268330+0000","last_clean":"2024-03-16T20:45:46.268330+0000","last_became_active":"2024-03-16T20:45:44.986860+0000","last_became_peered":"2024-03-16T20:45:44.986860+0000","last_unstale":"2024-03-16T20:45:46.268330+0000","last_undegraded":"2024-03-16T20:45:46.268330+0000","last_fullsized":"2024-03-16T20:45:46.268330+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:31:14.070831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956478+0000","last_change":"2024-03-16T20:45:44.997710+0000","last_active":"2024-03-16T20:45:45.956478+0000","last_peered":"2024-03-16T20:45:45.956478+0000","last_clean":"2024-03-16T20:45:45.956478+0000","last_became_active":"2024-03-16T20:45:44.996999+0000","last_became_peered":"2024-03-16T20:45:44.996999+0000","last_unstale":"2024-03-16T20:45:45.956478+0000","last_undegraded":"2024-03-16T20:45:45.956478+0000","last_fullsized":"2024-03-16T20:45:45.956478+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:40:04.897827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957631+0000","last_change":"2024-03-16T20:45:44.971505+0000","last_active":"2024-03-16T20:45:45.957631+0000","last_peered":"2024-03-16T20:45:45.957631+0000","last_clean":"2024-03-16T20:45:45.957631+0000","last_became_active":"2024-03-16T20:45:44.970737+0000","last_became_peered":"2024-03-16T20:45:44.970737+0000","last_unstale":"2024-03-16T20:45:45.957631+0000","last_undegraded":"2024-03-16T20:45:45.957631+0000","last_fullsized":"2024-03-16T20:45:45.957631+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:06:36.112288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269089+0000","last_change":"2024-03-16T20:45:44.988896+0000","last_active":"2024-03-16T20:45:46.269089+0000","last_peered":"2024-03-16T20:45:46.269089+0000","last_clean":"2024-03-16T20:45:46.269089+0000","last_became_active":"2024-03-16T20:45:44.988250+0000","last_became_peered":"2024-03-16T20:45:44.988250+0000","last_unstale":"2024-03-16T20:45:46.269089+0000","last_undegraded":"2024-03-16T20:45:46.269089+0000","last_fullsized":"2024-03-16T20:45:46.269089+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:57:53.975463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956681+0000","last_change":"2024-03-16T20:45:44.969082+0000","last_active":"2024-03-16T20:45:45.956681+0000","last_peered":"2024-03-16T20:45:45.956681+0000","last_clean":"2024-03-16T20:45:45.956681+0000","last_became_active":"2024-03-16T20:45:44.968282+0000","last_became_peered":"2024-03-16T20:45:44.968282+0000","last_unstale":"2024-03-16T20:45:45.956681+0000","last_undegraded":"2024-03-16T20:45:45.956681+0000","last_fullsized":"2024-03-16T20:45:45.956681+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:05:57.364887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267692+0000","last_change":"2024-03-16T20:45:44.984346+0000","last_active":"2024-03-16T20:45:46.267692+0000","last_peered":"2024-03-16T20:45:46.267692+0000","last_clean":"2024-03-16T20:45:46.267692+0000","last_became_active":"2024-03-16T20:45:44.983837+0000","last_became_peered":"2024-03-16T20:45:44.983837+0000","last_unstale":"2024-03-16T20:45:46.267692+0000","last_undegraded":"2024-03-16T20:45:46.267692+0000","last_fullsized":"2024-03-16T20:45:46.267692+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:46:10.557343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266735+0000","last_change":"2024-03-16T20:45:44.969608+0000","last_active":"2024-03-16T20:45:46.266735+0000","last_peered":"2024-03-16T20:45:46.266735+0000","last_clean":"2024-03-16T20:45:46.266735+0000","last_became_active":"2024-03-16T20:45:44.968876+0000","last_became_peered":"2024-03-16T20:45:44.968876+0000","last_unstale":"2024-03-16T20:45:46.266735+0000","last_undegraded":"2024-03-16T20:45:46.266735+0000","last_fullsized":"2024-03-16T20:45:46.266735+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:20:45.687785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.955788+0000","last_change":"2024-03-16T20:45:44.975592+0000","last_active":"2024-03-16T20:45:45.955788+0000","last_peered":"2024-03-16T20:45:45.955788+0000","last_clean":"2024-03-16T20:45:45.955788+0000","last_became_active":"2024-03-16T20:45:44.973733+0000","last_became_peered":"2024-03-16T20:45:44.973733+0000","last_unstale":"2024-03-16T20:45:45.955788+0000","last_undegraded":"2024-03-16T20:45:45.955788+0000","last_fullsized":"2024-03-16T20:45:45.955788+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:31:14.070831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269494+0000","last_change":"2024-03-16T20:45:44.976648+0000","last_active":"2024-03-16T20:45:46.269494+0000","last_peered":"2024-03-16T20:45:46.269494+0000","last_clean":"2024-03-16T20:45:46.269494+0000","last_became_active":"2024-03-16T20:45:44.975885+0000","last_became_peered":"2024-03-16T20:45:44.975885+0000","last_unstale":"2024-03-16T20:45:46.269494+0000","last_undegraded":"2024-03-16T20:45:46.269494+0000","last_fullsized":"2024-03-16T20:45:46.269494+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:08:10.036696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268613+0000","last_change":"2024-03-16T20:45:44.983529+0000","last_active":"2024-03-16T20:45:46.268613+0000","last_peered":"2024-03-16T20:45:46.268613+0000","last_clean":"2024-03-16T20:45:46.268613+0000","last_became_active":"2024-03-16T20:45:44.982855+0000","last_became_peered":"2024-03-16T20:45:44.982855+0000","last_unstale":"2024-03-16T20:45:46.268613+0000","last_undegraded":"2024-03-16T20:45:46.268613+0000","last_fullsized":"2024-03-16T20:45:46.268613+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:45:12.371596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266734+0000","last_change":"2024-03-16T20:45:44.986663+0000","last_active":"2024-03-16T20:45:46.266734+0000","last_peered":"2024-03-16T20:45:46.266734+0000","last_clean":"2024-03-16T20:45:46.266734+0000","last_became_active":"2024-03-16T20:45:44.985802+0000","last_became_peered":"2024-03-16T20:45:44.985802+0000","last_unstale":"2024-03-16T20:45:46.266734+0000","last_undegraded":"2024-03-16T20:45:46.266734+0000","last_fullsized":"2024-03-16T20:45:46.266734+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:29:17.211554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266824+0000","last_change":"2024-03-16T20:45:44.971503+0000","last_active":"2024-03-16T20:45:46.266824+0000","last_peered":"2024-03-16T20:45:46.266824+0000","last_clean":"2024-03-16T20:45:46.266824+0000","last_became_active":"2024-03-16T20:45:44.970764+0000","last_became_peered":"2024-03-16T20:45:44.970764+0000","last_unstale":"2024-03-16T20:45:46.266824+0000","last_undegraded":"2024-03-16T20:45:46.266824+0000","last_fullsized":"2024-03-16T20:45:46.266824+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:51:07.991442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267798+0000","last_change":"2024-03-16T20:45:44.981615+0000","last_active":"2024-03-16T20:45:46.267798+0000","last_peered":"2024-03-16T20:45:46.267798+0000","last_clean":"2024-03-16T20:45:46.267798+0000","last_became_active":"2024-03-16T20:45:44.980565+0000","last_became_peered":"2024-03-16T20:45:44.980565+0000","last_unstale":"2024-03-16T20:45:46.267798+0000","last_undegraded":"2024-03-16T20:45:46.267798+0000","last_fullsized":"2024-03-16T20:45:46.267798+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:42:10.851650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957994+0000","last_change":"2024-03-16T20:45:45.001667+0000","last_active":"2024-03-16T20:45:45.957994+0000","last_peered":"2024-03-16T20:45:45.957994+0000","last_clean":"2024-03-16T20:45:45.957994+0000","last_became_active":"2024-03-16T20:45:44.999133+0000","last_became_peered":"2024-03-16T20:45:44.999133+0000","last_unstale":"2024-03-16T20:45:45.957994+0000","last_undegraded":"2024-03-16T20:45:45.957994+0000","last_fullsized":"2024-03-16T20:45:45.957994+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:45:23.479155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956254+0000","last_change":"2024-03-16T20:45:44.996905+0000","last_active":"2024-03-16T20:45:45.956254+0000","last_peered":"2024-03-16T20:45:45.956254+0000","last_clean":"2024-03-16T20:45:45.956254+0000","last_became_active":"2024-03-16T20:45:44.996182+0000","last_became_peered":"2024-03-16T20:45:44.996182+0000","last_unstale":"2024-03-16T20:45:45.956254+0000","last_undegraded":"2024-03-16T20:45:45.956254+0000","last_fullsized":"2024-03-16T20:45:45.956254+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:41:27.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005074+0000","last_change":"2024-03-16T20:45:44.972167+0000","last_active":"2024-03-16T20:45:46.005074+0000","last_peered":"2024-03-16T20:45:46.005074+0000","last_clean":"2024-03-16T20:45:46.005074+0000","last_became_active":"2024-03-16T20:45:44.971283+0000","last_became_peered":"2024-03-16T20:45:44.971283+0000","last_unstale":"2024-03-16T20:45:46.005074+0000","last_undegraded":"2024-03-16T20:45:46.005074+0000","last_fullsized":"2024-03-16T20:45:46.005074+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:41:27.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.955247+0000","last_change":"2024-03-16T20:45:44.977536+0000","last_active":"2024-03-16T20:45:45.955247+0000","last_peered":"2024-03-16T20:45:45.955247+0000","last_clean":"2024-03-16T20:45:45.955247+0000","last_became_active":"2024-03-16T20:45:44.976794+0000","last_became_peered":"2024-03-16T20:45:44.976794+0000","last_unstale":"2024-03-16T20:45:45.955247+0000","last_undegraded":"2024-03-16T20:45:45.955247+0000","last_fullsized":"2024-03-16T20:45:45.955247+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:35:59.048311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957835+0000","last_change":"2024-03-16T20:45:44.976909+0000","last_active":"2024-03-16T20:45:45.957835+0000","last_peered":"2024-03-16T20:45:45.957835+0000","last_clean":"2024-03-16T20:45:45.957835+0000","last_became_active":"2024-03-16T20:45:44.976127+0000","last_became_peered":"2024-03-16T20:45:44.976127+0000","last_unstale":"2024-03-16T20:45:45.957835+0000","last_undegraded":"2024-03-16T20:45:45.957835+0000","last_fullsized":"2024-03-16T20:45:45.957835+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:45:12.371596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266763+0000","last_change":"2024-03-16T20:45:44.978761+0000","last_active":"2024-03-16T20:45:46.266763+0000","last_peered":"2024-03-16T20:45:46.266763+0000","last_clean":"2024-03-16T20:45:46.266763+0000","last_became_active":"2024-03-16T20:45:44.974520+0000","last_became_peered":"2024-03-16T20:45:44.974520+0000","last_unstale":"2024-03-16T20:45:46.266763+0000","last_undegraded":"2024-03-16T20:45:46.266763+0000","last_fullsized":"2024-03-16T20:45:46.266763+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:14:14.791268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957901+0000","last_change":"2024-03-16T20:45:45.004121+0000","last_active":"2024-03-16T20:45:45.957901+0000","last_peered":"2024-03-16T20:45:45.957901+0000","last_clean":"2024-03-16T20:45:45.957901+0000","last_became_active":"2024-03-16T20:45:45.003251+0000","last_became_peered":"2024-03-16T20:45:45.003251+0000","last_unstale":"2024-03-16T20:45:45.957901+0000","last_undegraded":"2024-03-16T20:45:45.957901+0000","last_fullsized":"2024-03-16T20:45:45.957901+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:39:41.263058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957018+0000","last_change":"2024-03-16T20:45:44.964295+0000","last_active":"2024-03-16T20:45:45.957018+0000","last_peered":"2024-03-16T20:45:45.957018+0000","last_clean":"2024-03-16T20:45:45.957018+0000","last_became_active":"2024-03-16T20:45:44.963502+0000","last_became_peered":"2024-03-16T20:45:44.963502+0000","last_unstale":"2024-03-16T20:45:45.957018+0000","last_undegraded":"2024-03-16T20:45:45.957018+0000","last_fullsized":"2024-03-16T20:45:45.957018+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:39:41.263058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957545+0000","last_change":"2024-03-16T20:45:45.004296+0000","last_active":"2024-03-16T20:45:45.957545+0000","last_peered":"2024-03-16T20:45:45.957545+0000","last_clean":"2024-03-16T20:45:45.957545+0000","last_became_active":"2024-03-16T20:45:44.997947+0000","last_became_peered":"2024-03-16T20:45:44.997947+0000","last_unstale":"2024-03-16T20:45:45.957545+0000","last_undegraded":"2024-03-16T20:45:45.957545+0000","last_fullsized":"2024-03-16T20:45:45.957545+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:25:32.739512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268596+0000","last_change":"2024-03-16T20:45:44.983629+0000","last_active":"2024-03-16T20:45:46.268596+0000","last_peered":"2024-03-16T20:45:46.268596+0000","last_clean":"2024-03-16T20:45:46.268596+0000","last_became_active":"2024-03-16T20:45:44.979161+0000","last_became_peered":"2024-03-16T20:45:44.979161+0000","last_unstale":"2024-03-16T20:45:46.268596+0000","last_undegraded":"2024-03-16T20:45:46.268596+0000","last_fullsized":"2024-03-16T20:45:46.268596+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:06:36.112288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267550+0000","last_change":"2024-03-16T20:45:44.977815+0000","last_active":"2024-03-16T20:45:46.267550+0000","last_peered":"2024-03-16T20:45:46.267550+0000","last_clean":"2024-03-16T20:45:46.267550+0000","last_became_active":"2024-03-16T20:45:44.975655+0000","last_became_peered":"2024-03-16T20:45:44.975655+0000","last_unstale":"2024-03-16T20:45:46.267550+0000","last_undegraded":"2024-03-16T20:45:46.267550+0000","last_fullsized":"2024-03-16T20:45:46.267550+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:30:32.602019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268765+0000","last_change":"2024-03-16T20:45:45.003806+0000","last_active":"2024-03-16T20:45:46.268765+0000","last_peered":"2024-03-16T20:45:46.268765+0000","last_clean":"2024-03-16T20:45:46.268765+0000","last_became_active":"2024-03-16T20:45:45.002935+0000","last_became_peered":"2024-03-16T20:45:45.002935+0000","last_unstale":"2024-03-16T20:45:46.268765+0000","last_undegraded":"2024-03-16T20:45:46.268765+0000","last_fullsized":"2024-03-16T20:45:46.268765+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:49.546012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005668+0000","last_change":"2024-03-16T20:45:44.977311+0000","last_active":"2024-03-16T20:45:46.005668+0000","last_peered":"2024-03-16T20:45:46.005668+0000","last_clean":"2024-03-16T20:45:46.005668+0000","last_became_active":"2024-03-16T20:45:44.975825+0000","last_became_peered":"2024-03-16T20:45:44.975825+0000","last_unstale":"2024-03-16T20:45:46.005668+0000","last_undegraded":"2024-03-16T20:45:46.005668+0000","last_fullsized":"2024-03-16T20:45:46.005668+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:25:32.739512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"29'4","reported_seq":18,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269202+0000","last_change":"2024-03-16T20:45:43.956395+0000","last_active":"2024-03-16T20:45:46.269202+0000","last_peered":"2024-03-16T20:45:46.269202+0000","last_clean":"2024-03-16T20:45:46.269202+0000","last_became_active":"2024-03-16T20:45:43.955548+0000","last_became_peered":"2024-03-16T20:45:43.955548+0000","last_unstale":"2024-03-16T20:45:46.269202+0000","last_undegraded":"2024-03-16T20:45:46.269202+0000","last_fullsized":"2024-03-16T20:45:46.269202+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:29:54.412142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268904+0000","last_change":"2024-03-16T20:45:45.001424+0000","last_active":"2024-03-16T20:45:46.268904+0000","last_peered":"2024-03-16T20:45:46.268904+0000","last_clean":"2024-03-16T20:45:46.268904+0000","last_became_active":"2024-03-16T20:45:45.000845+0000","last_became_peered":"2024-03-16T20:45:45.000845+0000","last_unstale":"2024-03-16T20:45:46.268904+0000","last_undegraded":"2024-03-16T20:45:46.268904+0000","last_fullsized":"2024-03-16T20:45:46.268904+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:57:51.937834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005625+0000","last_change":"2024-03-16T20:45:43.941139+0000","last_active":"2024-03-16T20:45:46.005625+0000","last_peered":"2024-03-16T20:45:46.005625+0000","last_clean":"2024-03-16T20:45:46.005625+0000","last_became_active":"2024-03-16T20:45:43.940339+0000","last_became_peered":"2024-03-16T20:45:43.940339+0000","last_unstale":"2024-03-16T20:45:46.005625+0000","last_undegraded":"2024-03-16T20:45:46.005625+0000","last_fullsized":"2024-03-16T20:45:46.005625+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:11:09.879241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956868+0000","last_change":"2024-03-16T20:45:44.999396+0000","last_active":"2024-03-16T20:45:45.956868+0000","last_peered":"2024-03-16T20:45:45.956868+0000","last_clean":"2024-03-16T20:45:45.956868+0000","last_became_active":"2024-03-16T20:45:44.997352+0000","last_became_peered":"2024-03-16T20:45:44.997352+0000","last_unstale":"2024-03-16T20:45:45.956868+0000","last_undegraded":"2024-03-16T20:45:45.956868+0000","last_fullsized":"2024-03-16T20:45:45.956868+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:57:51.937834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266929+0000","last_change":"2024-03-16T20:45:43.932198+0000","last_active":"2024-03-16T20:45:46.266929+0000","last_peered":"2024-03-16T20:45:46.266929+0000","last_clean":"2024-03-16T20:45:46.266929+0000","last_became_active":"2024-03-16T20:45:43.931513+0000","last_became_peered":"2024-03-16T20:45:43.931513+0000","last_unstale":"2024-03-16T20:45:46.266929+0000","last_undegraded":"2024-03-16T20:45:46.266929+0000","last_fullsized":"2024-03-16T20:45:46.266929+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:30:31.593443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957955+0000","last_change":"2024-03-16T20:45:45.003785+0000","last_active":"2024-03-16T20:45:45.957955+0000","last_peered":"2024-03-16T20:45:45.957955+0000","last_clean":"2024-03-16T20:45:45.957955+0000","last_became_active":"2024-03-16T20:45:45.003497+0000","last_became_peered":"2024-03-16T20:45:45.003497+0000","last_unstale":"2024-03-16T20:45:45.957955+0000","last_undegraded":"2024-03-16T20:45:45.957955+0000","last_fullsized":"2024-03-16T20:45:45.957955+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:01:49.546012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267880+0000","last_change":"2024-03-16T20:45:43.929779+0000","last_active":"2024-03-16T20:45:46.267880+0000","last_peered":"2024-03-16T20:45:46.267880+0000","last_clean":"2024-03-16T20:45:46.267880+0000","last_became_active":"2024-03-16T20:45:43.929496+0000","last_became_peered":"2024-03-16T20:45:43.929496+0000","last_unstale":"2024-03-16T20:45:46.267880+0000","last_undegraded":"2024-03-16T20:45:46.267880+0000","last_fullsized":"2024-03-16T20:45:46.267880+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:35:46.461735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268734+0000","last_change":"2024-03-16T20:45:45.000918+0000","last_active":"2024-03-16T20:45:46.268734+0000","last_peered":"2024-03-16T20:45:46.268734+0000","last_clean":"2024-03-16T20:45:46.268734+0000","last_became_active":"2024-03-16T20:45:45.000298+0000","last_became_peered":"2024-03-16T20:45:45.000298+0000","last_unstale":"2024-03-16T20:45:46.268734+0000","last_undegraded":"2024-03-16T20:45:46.268734+0000","last_fullsized":"2024-03-16T20:45:46.268734+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:39:41.263058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957811+0000","last_change":"2024-03-16T20:45:43.942190+0000","last_active":"2024-03-16T20:45:45.957811+0000","last_peered":"2024-03-16T20:45:45.957811+0000","last_clean":"2024-03-16T20:45:45.957811+0000","last_became_active":"2024-03-16T20:45:43.938832+0000","last_became_peered":"2024-03-16T20:45:43.938832+0000","last_unstale":"2024-03-16T20:45:45.957811+0000","last_undegraded":"2024-03-16T20:45:45.957811+0000","last_fullsized":"2024-03-16T20:45:45.957811+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:04:09.454448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957353+0000","last_change":"2024-03-16T20:45:44.966859+0000","last_active":"2024-03-16T20:45:45.957353+0000","last_peered":"2024-03-16T20:45:45.957353+0000","last_clean":"2024-03-16T20:45:45.957353+0000","last_became_active":"2024-03-16T20:45:44.966087+0000","last_became_peered":"2024-03-16T20:45:44.966087+0000","last_unstale":"2024-03-16T20:45:45.957353+0000","last_undegraded":"2024-03-16T20:45:45.957353+0000","last_fullsized":"2024-03-16T20:45:45.957353+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:57:53.975463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956556+0000","last_change":"2024-03-16T20:45:43.935054+0000","last_active":"2024-03-16T20:45:45.956556+0000","last_peered":"2024-03-16T20:45:45.956556+0000","last_clean":"2024-03-16T20:45:45.956556+0000","last_became_active":"2024-03-16T20:45:43.933153+0000","last_became_peered":"2024-03-16T20:45:43.933153+0000","last_unstale":"2024-03-16T20:45:45.956556+0000","last_undegraded":"2024-03-16T20:45:45.956556+0000","last_fullsized":"2024-03-16T20:45:45.956556+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:05:20.864966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.955670+0000","last_change":"2024-03-16T20:45:44.997096+0000","last_active":"2024-03-16T20:45:45.955670+0000","last_peered":"2024-03-16T20:45:45.955670+0000","last_clean":"2024-03-16T20:45:45.955670+0000","last_became_active":"2024-03-16T20:45:44.996219+0000","last_became_peered":"2024-03-16T20:45:44.996219+0000","last_unstale":"2024-03-16T20:45:45.955670+0000","last_undegraded":"2024-03-16T20:45:45.955670+0000","last_fullsized":"2024-03-16T20:45:45.955670+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:59:40.370844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266166+0000","last_change":"2024-03-16T20:45:43.929969+0000","last_active":"2024-03-16T20:45:46.266166+0000","last_peered":"2024-03-16T20:45:46.266166+0000","last_clean":"2024-03-16T20:45:46.266166+0000","last_became_active":"2024-03-16T20:45:43.929709+0000","last_became_peered":"2024-03-16T20:45:43.929709+0000","last_unstale":"2024-03-16T20:45:46.266166+0000","last_undegraded":"2024-03-16T20:45:46.266166+0000","last_fullsized":"2024-03-16T20:45:46.266166+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:07:47.365995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956485+0000","last_change":"2024-03-16T20:45:44.969933+0000","last_active":"2024-03-16T20:45:45.956485+0000","last_peered":"2024-03-16T20:45:45.956485+0000","last_clean":"2024-03-16T20:45:45.956485+0000","last_became_active":"2024-03-16T20:45:44.969188+0000","last_became_peered":"2024-03-16T20:45:44.969188+0000","last_unstale":"2024-03-16T20:45:45.956485+0000","last_undegraded":"2024-03-16T20:45:45.956485+0000","last_fullsized":"2024-03-16T20:45:45.956485+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:35:59.048311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956640+0000","last_change":"2024-03-16T20:45:43.937169+0000","last_active":"2024-03-16T20:45:45.956640+0000","last_peered":"2024-03-16T20:45:45.956640+0000","last_clean":"2024-03-16T20:45:45.956640+0000","last_became_active":"2024-03-16T20:45:43.936408+0000","last_became_peered":"2024-03-16T20:45:43.936408+0000","last_unstale":"2024-03-16T20:45:45.956640+0000","last_undegraded":"2024-03-16T20:45:45.956640+0000","last_fullsized":"2024-03-16T20:45:45.956640+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:58:24.502319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957020+0000","last_change":"2024-03-16T20:45:44.974971+0000","last_active":"2024-03-16T20:45:45.957020+0000","last_peered":"2024-03-16T20:45:45.957020+0000","last_clean":"2024-03-16T20:45:45.957020+0000","last_became_active":"2024-03-16T20:45:44.973981+0000","last_became_peered":"2024-03-16T20:45:44.973981+0000","last_unstale":"2024-03-16T20:45:45.957020+0000","last_undegraded":"2024-03-16T20:45:45.957020+0000","last_fullsized":"2024-03-16T20:45:45.957020+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:45:12.371596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267024+0000","last_change":"2024-03-16T20:45:43.948526+0000","last_active":"2024-03-16T20:45:46.267024+0000","last_peered":"2024-03-16T20:45:46.267024+0000","last_clean":"2024-03-16T20:45:46.267024+0000","last_became_active":"2024-03-16T20:45:43.948226+0000","last_became_peered":"2024-03-16T20:45:43.948226+0000","last_unstale":"2024-03-16T20:45:46.267024+0000","last_undegraded":"2024-03-16T20:45:46.267024+0000","last_fullsized":"2024-03-16T20:45:46.267024+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:40:03.889251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266987+0000","last_change":"2024-03-16T20:45:44.986140+0000","last_active":"2024-03-16T20:45:46.266987+0000","last_peered":"2024-03-16T20:45:46.266987+0000","last_clean":"2024-03-16T20:45:46.266987+0000","last_became_active":"2024-03-16T20:45:44.985505+0000","last_became_peered":"2024-03-16T20:45:44.985505+0000","last_unstale":"2024-03-16T20:45:46.266987+0000","last_undegraded":"2024-03-16T20:45:46.266987+0000","last_fullsized":"2024-03-16T20:45:46.266987+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:04:57.201475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005726+0000","last_change":"2024-03-16T20:45:43.930479+0000","last_active":"2024-03-16T20:45:46.005726+0000","last_peered":"2024-03-16T20:45:46.005726+0000","last_clean":"2024-03-16T20:45:46.005726+0000","last_became_active":"2024-03-16T20:45:43.929953+0000","last_became_peered":"2024-03-16T20:45:43.929953+0000","last_unstale":"2024-03-16T20:45:46.005726+0000","last_undegraded":"2024-03-16T20:45:46.005726+0000","last_fullsized":"2024-03-16T20:45:46.005726+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:52:52.971592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267140+0000","last_change":"2024-03-16T20:45:44.976577+0000","last_active":"2024-03-16T20:45:46.267140+0000","last_peered":"2024-03-16T20:45:46.267140+0000","last_clean":"2024-03-16T20:45:46.267140+0000","last_became_active":"2024-03-16T20:45:44.975824+0000","last_became_peered":"2024-03-16T20:45:44.975824+0000","last_unstale":"2024-03-16T20:45:46.267140+0000","last_undegraded":"2024-03-16T20:45:46.267140+0000","last_fullsized":"2024-03-16T20:45:46.267140+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:39:41.772078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"29'3","reported_seq":17,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957551+0000","last_change":"2024-03-16T20:45:43.955812+0000","last_active":"2024-03-16T20:45:45.957551+0000","last_peered":"2024-03-16T20:45:45.957551+0000","last_clean":"2024-03-16T20:45:45.957551+0000","last_became_active":"2024-03-16T20:45:43.955052+0000","last_became_peered":"2024-03-16T20:45:43.955052+0000","last_unstale":"2024-03-16T20:45:45.957551+0000","last_undegraded":"2024-03-16T20:45:45.957551+0000","last_fullsized":"2024-03-16T20:45:45.957551+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:29:16.261608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956689+0000","last_change":"2024-03-16T20:45:44.985675+0000","last_active":"2024-03-16T20:45:45.956689+0000","last_peered":"2024-03-16T20:45:45.956689+0000","last_clean":"2024-03-16T20:45:45.956689+0000","last_became_active":"2024-03-16T20:45:44.984881+0000","last_became_peered":"2024-03-16T20:45:44.984881+0000","last_unstale":"2024-03-16T20:45:45.956689+0000","last_undegraded":"2024-03-16T20:45:45.956689+0000","last_fullsized":"2024-03-16T20:45:45.956689+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:06:36.112288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"29'6","reported_seq":20,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267085+0000","last_change":"2024-03-16T20:45:43.932131+0000","last_active":"2024-03-16T20:45:46.267085+0000","last_peered":"2024-03-16T20:45:46.267085+0000","last_clean":"2024-03-16T20:45:46.267085+0000","last_became_active":"2024-03-16T20:45:43.931603+0000","last_became_peered":"2024-03-16T20:45:43.931603+0000","last_unstale":"2024-03-16T20:45:46.267085+0000","last_undegraded":"2024-03-16T20:45:46.267085+0000","last_fullsized":"2024-03-16T20:45:46.267085+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:25:31.730936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956486+0000","last_change":"2024-03-16T20:45:44.974010+0000","last_active":"2024-03-16T20:45:45.956486+0000","last_peered":"2024-03-16T20:45:45.956486+0000","last_clean":"2024-03-16T20:45:45.956486+0000","last_became_active":"2024-03-16T20:45:44.973236+0000","last_became_peered":"2024-03-16T20:45:44.973236+0000","last_unstale":"2024-03-16T20:45:45.956486+0000","last_undegraded":"2024-03-16T20:45:45.956486+0000","last_fullsized":"2024-03-16T20:45:45.956486+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:39:42.030933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.958082+0000","last_change":"2024-03-16T20:45:43.940540+0000","last_active":"2024-03-16T20:45:45.958082+0000","last_peered":"2024-03-16T20:45:45.958082+0000","last_clean":"2024-03-16T20:45:45.958082+0000","last_became_active":"2024-03-16T20:45:43.939463+0000","last_became_peered":"2024-03-16T20:45:43.939463+0000","last_unstale":"2024-03-16T20:45:45.958082+0000","last_undegraded":"2024-03-16T20:45:45.958082+0000","last_fullsized":"2024-03-16T20:45:45.958082+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:46:09.548766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266648+0000","last_change":"2024-03-16T20:45:44.967457+0000","last_active":"2024-03-16T20:45:46.266648+0000","last_peered":"2024-03-16T20:45:46.266648+0000","last_clean":"2024-03-16T20:45:46.266648+0000","last_became_active":"2024-03-16T20:45:44.966729+0000","last_became_peered":"2024-03-16T20:45:44.966729+0000","last_unstale":"2024-03-16T20:45:46.266648+0000","last_undegraded":"2024-03-16T20:45:46.266648+0000","last_fullsized":"2024-03-16T20:45:46.266648+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:20:42.752303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956344+0000","last_change":"2024-03-16T20:45:43.936392+0000","last_active":"2024-03-16T20:45:45.956344+0000","last_peered":"2024-03-16T20:45:45.956344+0000","last_clean":"2024-03-16T20:45:45.956344+0000","last_became_active":"2024-03-16T20:45:43.936050+0000","last_became_peered":"2024-03-16T20:45:43.936050+0000","last_unstale":"2024-03-16T20:45:45.956344+0000","last_undegraded":"2024-03-16T20:45:45.956344+0000","last_fullsized":"2024-03-16T20:45:45.956344+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:29:16.261608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269169+0000","last_change":"2024-03-16T20:45:45.002182+0000","last_active":"2024-03-16T20:45:46.269169+0000","last_peered":"2024-03-16T20:45:46.269169+0000","last_clean":"2024-03-16T20:45:46.269169+0000","last_became_active":"2024-03-16T20:45:45.000789+0000","last_became_peered":"2024-03-16T20:45:45.000789+0000","last_unstale":"2024-03-16T20:45:46.269169+0000","last_undegraded":"2024-03-16T20:45:46.269169+0000","last_fullsized":"2024-03-16T20:45:46.269169+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:02:16.196327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266267+0000","last_change":"2024-03-16T20:45:43.947592+0000","last_active":"2024-03-16T20:45:46.266267+0000","last_peered":"2024-03-16T20:45:46.266267+0000","last_clean":"2024-03-16T20:45:46.266267+0000","last_became_active":"2024-03-16T20:45:43.947229+0000","last_became_peered":"2024-03-16T20:45:43.947229+0000","last_unstale":"2024-03-16T20:45:46.266267+0000","last_undegraded":"2024-03-16T20:45:46.266267+0000","last_fullsized":"2024-03-16T20:45:46.266267+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:41:26.046777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267667+0000","last_change":"2024-03-16T20:45:45.002016+0000","last_active":"2024-03-16T20:45:46.267667+0000","last_peered":"2024-03-16T20:45:46.267667+0000","last_clean":"2024-03-16T20:45:46.267667+0000","last_became_active":"2024-03-16T20:45:45.001286+0000","last_became_peered":"2024-03-16T20:45:45.001286+0000","last_unstale":"2024-03-16T20:45:46.267667+0000","last_undegraded":"2024-03-16T20:45:46.267667+0000","last_fullsized":"2024-03-16T20:45:46.267667+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:07:48.374571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"29'8","reported_seq":22,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266238+0000","last_change":"2024-03-16T20:45:43.932897+0000","last_active":"2024-03-16T20:45:46.266238+0000","last_peered":"2024-03-16T20:45:46.266238+0000","last_clean":"2024-03-16T20:45:46.266238+0000","last_became_active":"2024-03-16T20:45:43.931665+0000","last_became_peered":"2024-03-16T20:45:43.931665+0000","last_unstale":"2024-03-16T20:45:46.266238+0000","last_undegraded":"2024-03-16T20:45:46.266238+0000","last_fullsized":"2024-03-16T20:45:46.266238+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:45:22.470578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956126+0000","last_change":"2024-03-16T20:45:44.999741+0000","last_active":"2024-03-16T20:45:45.956126+0000","last_peered":"2024-03-16T20:45:45.956126+0000","last_clean":"2024-03-16T20:45:45.956126+0000","last_became_active":"2024-03-16T20:45:44.998998+0000","last_became_peered":"2024-03-16T20:45:44.998998+0000","last_unstale":"2024-03-16T20:45:45.956126+0000","last_undegraded":"2024-03-16T20:45:45.956126+0000","last_fullsized":"2024-03-16T20:45:45.956126+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T01:07:48.374571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.958152+0000","last_change":"2024-03-16T20:45:43.932373+0000","last_active":"2024-03-16T20:45:45.958152+0000","last_peered":"2024-03-16T20:45:45.958152+0000","last_clean":"2024-03-16T20:45:45.958152+0000","last_became_active":"2024-03-16T20:45:43.931593+0000","last_became_peered":"2024-03-16T20:45:43.931593+0000","last_unstale":"2024-03-16T20:45:45.958152+0000","last_undegraded":"2024-03-16T20:45:45.958152+0000","last_fullsized":"2024-03-16T20:45:45.958152+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:29:54.412142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957975+0000","last_change":"2024-03-16T20:45:45.003331+0000","last_active":"2024-03-16T20:45:45.957975+0000","last_peered":"2024-03-16T20:45:45.957975+0000","last_clean":"2024-03-16T20:45:45.957975+0000","last_became_active":"2024-03-16T20:45:45.002518+0000","last_became_peered":"2024-03-16T20:45:45.002518+0000","last_unstale":"2024-03-16T20:45:45.957975+0000","last_undegraded":"2024-03-16T20:45:45.957975+0000","last_fullsized":"2024-03-16T20:45:45.957975+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:02:16.196327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956428+0000","last_change":"2024-03-16T20:45:43.942493+0000","last_active":"2024-03-16T20:45:45.956428+0000","last_peered":"2024-03-16T20:45:45.956428+0000","last_clean":"2024-03-16T20:45:45.956428+0000","last_became_active":"2024-03-16T20:45:43.941761+0000","last_became_peered":"2024-03-16T20:45:43.941761+0000","last_unstale":"2024-03-16T20:45:45.956428+0000","last_undegraded":"2024-03-16T20:45:45.956428+0000","last_fullsized":"2024-03-16T20:45:45.956428+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:59:41.199216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957015+0000","last_change":"2024-03-16T20:45:45.002530+0000","last_active":"2024-03-16T20:45:45.957015+0000","last_peered":"2024-03-16T20:45:45.957015+0000","last_clean":"2024-03-16T20:45:45.957015+0000","last_became_active":"2024-03-16T20:45:45.002209+0000","last_became_peered":"2024-03-16T20:45:45.002209+0000","last_unstale":"2024-03-16T20:45:45.957015+0000","last_undegraded":"2024-03-16T20:45:45.957015+0000","last_fullsized":"2024-03-16T20:45:45.957015+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:31:14.070831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"29'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956922+0000","last_change":"2024-03-16T20:45:43.928863+0000","last_active":"2024-03-16T20:45:45.956922+0000","last_peered":"2024-03-16T20:45:45.956922+0000","last_clean":"2024-03-16T20:45:45.956922+0000","last_became_active":"2024-03-16T20:45:43.928073+0000","last_became_peered":"2024-03-16T20:45:43.928073+0000","last_unstale":"2024-03-16T20:45:45.956922+0000","last_undegraded":"2024-03-16T20:45:45.956922+0000","last_fullsized":"2024-03-16T20:45:45.956922+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:54:33.705576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267925+0000","last_change":"2024-03-16T20:45:44.976132+0000","last_active":"2024-03-16T20:45:46.267925+0000","last_peered":"2024-03-16T20:45:46.267925+0000","last_clean":"2024-03-16T20:45:46.267925+0000","last_became_active":"2024-03-16T20:45:44.975425+0000","last_became_peered":"2024-03-16T20:45:44.975425+0000","last_unstale":"2024-03-16T20:45:46.267925+0000","last_undegraded":"2024-03-16T20:45:46.267925+0000","last_fullsized":"2024-03-16T20:45:46.267925+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:04:10.463024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957127+0000","last_change":"2024-03-16T20:45:43.933789+0000","last_active":"2024-03-16T20:45:45.957127+0000","last_peered":"2024-03-16T20:45:45.957127+0000","last_clean":"2024-03-16T20:45:45.957127+0000","last_became_active":"2024-03-16T20:45:43.933013+0000","last_became_peered":"2024-03-16T20:45:43.933013+0000","last_unstale":"2024-03-16T20:45:45.957127+0000","last_undegraded":"2024-03-16T20:45:45.957127+0000","last_fullsized":"2024-03-16T20:45:45.957127+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:42:09.843074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267621+0000","last_change":"2024-03-16T20:45:45.000163+0000","last_active":"2024-03-16T20:45:46.267621+0000","last_peered":"2024-03-16T20:45:46.267621+0000","last_clean":"2024-03-16T20:45:46.267621+0000","last_became_active":"2024-03-16T20:45:44.999781+0000","last_became_peered":"2024-03-16T20:45:44.999781+0000","last_unstale":"2024-03-16T20:45:46.267621+0000","last_undegraded":"2024-03-16T20:45:46.267621+0000","last_fullsized":"2024-03-16T20:45:46.267621+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T07:58:25.510895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005354+0000","last_change":"2024-03-16T20:45:43.934020+0000","last_active":"2024-03-16T20:45:46.005354+0000","last_peered":"2024-03-16T20:45:46.005354+0000","last_clean":"2024-03-16T20:45:46.005354+0000","last_became_active":"2024-03-16T20:45:43.933382+0000","last_became_peered":"2024-03-16T20:45:43.933382+0000","last_unstale":"2024-03-16T20:45:46.005354+0000","last_undegraded":"2024-03-16T20:45:46.005354+0000","last_fullsized":"2024-03-16T20:45:46.005354+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:07:33.678359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267811+0000","last_change":"2024-03-16T20:45:44.974563+0000","last_active":"2024-03-16T20:45:46.267811+0000","last_peered":"2024-03-16T20:45:46.267811+0000","last_clean":"2024-03-16T20:45:46.267811+0000","last_became_active":"2024-03-16T20:45:44.968061+0000","last_became_peered":"2024-03-16T20:45:44.968061+0000","last_unstale":"2024-03-16T20:45:46.267811+0000","last_undegraded":"2024-03-16T20:45:46.267811+0000","last_fullsized":"2024-03-16T20:45:46.267811+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:54:34.714152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268258+0000","last_change":"2024-03-16T20:45:43.938395+0000","last_active":"2024-03-16T20:45:46.268258+0000","last_peered":"2024-03-16T20:45:46.268258+0000","last_clean":"2024-03-16T20:45:46.268258+0000","last_became_active":"2024-03-16T20:45:43.937726+0000","last_became_peered":"2024-03-16T20:45:43.937726+0000","last_unstale":"2024-03-16T20:45:46.268258+0000","last_undegraded":"2024-03-16T20:45:46.268258+0000","last_fullsized":"2024-03-16T20:45:46.268258+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:07:33.678359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267499+0000","last_change":"2024-03-16T20:45:44.976550+0000","last_active":"2024-03-16T20:45:46.267499+0000","last_peered":"2024-03-16T20:45:46.267499+0000","last_clean":"2024-03-16T20:45:46.267499+0000","last_became_active":"2024-03-16T20:45:44.975758+0000","last_became_peered":"2024-03-16T20:45:44.975758+0000","last_unstale":"2024-03-16T20:45:46.267499+0000","last_undegraded":"2024-03-16T20:45:46.267499+0000","last_fullsized":"2024-03-16T20:45:46.267499+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:19:27.805219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956027+0000","last_change":"2024-03-16T20:45:43.940698+0000","last_active":"2024-03-16T20:45:45.956027+0000","last_peered":"2024-03-16T20:45:45.956027+0000","last_clean":"2024-03-16T20:45:45.956027+0000","last_became_active":"2024-03-16T20:45:43.940118+0000","last_became_peered":"2024-03-16T20:45:43.940118+0000","last_unstale":"2024-03-16T20:45:45.956027+0000","last_undegraded":"2024-03-16T20:45:45.956027+0000","last_fullsized":"2024-03-16T20:45:45.956027+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:19:26.796643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269086+0000","last_change":"2024-03-16T20:45:44.974628+0000","last_active":"2024-03-16T20:45:46.269086+0000","last_peered":"2024-03-16T20:45:46.269086+0000","last_clean":"2024-03-16T20:45:46.269086+0000","last_became_active":"2024-03-16T20:45:44.973407+0000","last_became_peered":"2024-03-16T20:45:44.973407+0000","last_unstale":"2024-03-16T20:45:46.269086+0000","last_undegraded":"2024-03-16T20:45:46.269086+0000","last_fullsized":"2024-03-16T20:45:46.269086+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:59:19.229572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.955147+0000","last_change":"2024-03-16T20:45:43.952550+0000","last_active":"2024-03-16T20:45:45.955147+0000","last_peered":"2024-03-16T20:45:45.955147+0000","last_clean":"2024-03-16T20:45:45.955147+0000","last_became_active":"2024-03-16T20:45:43.951942+0000","last_became_peered":"2024-03-16T20:45:43.951942+0000","last_unstale":"2024-03-16T20:45:45.955147+0000","last_undegraded":"2024-03-16T20:45:45.955147+0000","last_fullsized":"2024-03-16T20:45:45.955147+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:52:52.971592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267666+0000","last_change":"2024-03-16T20:45:44.979710+0000","last_active":"2024-03-16T20:45:46.267666+0000","last_peered":"2024-03-16T20:45:46.267666+0000","last_clean":"2024-03-16T20:45:46.267666+0000","last_became_active":"2024-03-16T20:45:44.976874+0000","last_became_peered":"2024-03-16T20:45:44.976874+0000","last_unstale":"2024-03-16T20:45:46.267666+0000","last_undegraded":"2024-03-16T20:45:46.267666+0000","last_fullsized":"2024-03-16T20:45:46.267666+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T21:05:21.873542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"29'2","reported_seq":16,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.958317+0000","last_change":"2024-03-16T20:45:43.955309+0000","last_active":"2024-03-16T20:45:45.958317+0000","last_peered":"2024-03-16T20:45:45.958317+0000","last_clean":"2024-03-16T20:45:45.958317+0000","last_became_active":"2024-03-16T20:45:43.954978+0000","last_became_peered":"2024-03-16T20:45:43.954978+0000","last_unstale":"2024-03-16T20:45:45.958317+0000","last_undegraded":"2024-03-16T20:45:45.958317+0000","last_fullsized":"2024-03-16T20:45:45.958317+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:11:09.879241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957959+0000","last_change":"2024-03-16T20:45:44.984438+0000","last_active":"2024-03-16T20:45:45.957959+0000","last_peered":"2024-03-16T20:45:45.957959+0000","last_clean":"2024-03-16T20:45:45.957959+0000","last_became_active":"2024-03-16T20:45:44.983734+0000","last_became_peered":"2024-03-16T20:45:44.983734+0000","last_unstale":"2024-03-16T20:45:45.957959+0000","last_undegraded":"2024-03-16T20:45:45.957959+0000","last_fullsized":"2024-03-16T20:45:45.957959+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:59:19.229572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"30'1","reported_seq":15,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.007829+0000","last_change":"2024-03-16T20:45:43.930228+0000","last_active":"2024-03-16T20:45:46.007829+0000","last_peered":"2024-03-16T20:45:46.007829+0000","last_clean":"2024-03-16T20:45:46.007829+0000","last_became_active":"2024-03-16T20:45:43.929546+0000","last_became_peered":"2024-03-16T20:45:43.929546+0000","last_unstale":"2024-03-16T20:45:46.007829+0000","last_undegraded":"2024-03-16T20:45:46.007829+0000","last_fullsized":"2024-03-16T20:45:46.007829+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T02:29:16.261608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005194+0000","last_change":"2024-03-16T20:45:44.973355+0000","last_active":"2024-03-16T20:45:46.005194+0000","last_peered":"2024-03-16T20:45:46.005194+0000","last_clean":"2024-03-16T20:45:46.005194+0000","last_became_active":"2024-03-16T20:45:44.972472+0000","last_became_peered":"2024-03-16T20:45:44.972472+0000","last_unstale":"2024-03-16T20:45:46.005194+0000","last_undegraded":"2024-03-16T20:45:46.005194+0000","last_fullsized":"2024-03-16T20:45:46.005194+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T23:40:04.897827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.957197+0000","last_change":"2024-03-16T20:45:43.927656+0000","last_active":"2024-03-16T20:45:45.957197+0000","last_peered":"2024-03-16T20:45:45.957197+0000","last_clean":"2024-03-16T20:45:45.957197+0000","last_became_active":"2024-03-16T20:45:43.927393+0000","last_became_peered":"2024-03-16T20:45:43.927393+0000","last_unstale":"2024-03-16T20:45:45.957197+0000","last_undegraded":"2024-03-16T20:45:45.957197+0000","last_fullsized":"2024-03-16T20:45:45.957197+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:49:47.476324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267849+0000","last_change":"2024-03-16T20:45:44.986156+0000","last_active":"2024-03-16T20:45:46.267849+0000","last_peered":"2024-03-16T20:45:46.267849+0000","last_clean":"2024-03-16T20:45:46.267849+0000","last_became_active":"2024-03-16T20:45:44.983306+0000","last_became_peered":"2024-03-16T20:45:44.983306+0000","last_unstale":"2024-03-16T20:45:46.267849+0000","last_undegraded":"2024-03-16T20:45:46.267849+0000","last_fullsized":"2024-03-16T20:45:46.267849+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:41:27.055353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005447+0000","last_change":"2024-03-16T20:45:43.955611+0000","last_active":"2024-03-16T20:45:46.005447+0000","last_peered":"2024-03-16T20:45:46.005447+0000","last_clean":"2024-03-16T20:45:46.005447+0000","last_became_active":"2024-03-16T20:45:43.955171+0000","last_became_peered":"2024-03-16T20:45:43.955171+0000","last_unstale":"2024-03-16T20:45:46.005447+0000","last_undegraded":"2024-03-16T20:45:46.005447+0000","last_fullsized":"2024-03-16T20:45:46.005447+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T22:29:54.412142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.005252+0000","last_change":"2024-03-16T20:45:44.973307+0000","last_active":"2024-03-16T20:45:46.005252+0000","last_peered":"2024-03-16T20:45:46.005252+0000","last_clean":"2024-03-16T20:45:46.005252+0000","last_became_active":"2024-03-16T20:45:44.972319+0000","last_became_peered":"2024-03-16T20:45:44.972319+0000","last_unstale":"2024-03-16T20:45:46.005252+0000","last_undegraded":"2024-03-16T20:45:46.005252+0000","last_fullsized":"2024-03-16T20:45:46.005252+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:57:51.937834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956627+0000","last_change":"2024-03-16T20:45:43.955510+0000","last_active":"2024-03-16T20:45:45.956627+0000","last_peered":"2024-03-16T20:45:45.956627+0000","last_clean":"2024-03-16T20:45:45.956627+0000","last_became_active":"2024-03-16T20:45:43.955111+0000","last_became_peered":"2024-03-16T20:45:43.955111+0000","last_unstale":"2024-03-16T20:45:45.956627+0000","last_undegraded":"2024-03-16T20:45:45.956627+0000","last_fullsized":"2024-03-16T20:45:45.956627+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:52:52.971592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266225+0000","last_change":"2024-03-16T20:45:44.978908+0000","last_active":"2024-03-16T20:45:46.266225+0000","last_peered":"2024-03-16T20:45:46.266225+0000","last_clean":"2024-03-16T20:45:46.266225+0000","last_became_active":"2024-03-16T20:45:44.978200+0000","last_became_peered":"2024-03-16T20:45:44.978200+0000","last_unstale":"2024-03-16T20:45:46.266225+0000","last_undegraded":"2024-03-16T20:45:46.266225+0000","last_fullsized":"2024-03-16T20:45:46.266225+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T08:02:16.196327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.956274+0000","last_change":"2024-03-16T20:45:43.952698+0000","last_active":"2024-03-16T20:45:45.956274+0000","last_peered":"2024-03-16T20:45:45.956274+0000","last_clean":"2024-03-16T20:45:45.956274+0000","last_became_active":"2024-03-16T20:45:43.952125+0000","last_became_peered":"2024-03-16T20:45:43.952125+0000","last_unstale":"2024-03-16T20:45:45.956274+0000","last_undegraded":"2024-03-16T20:45:45.956274+0000","last_fullsized":"2024-03-16T20:45:45.956274+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T06:40:56.332196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:45.955744+0000","last_change":"2024-03-16T20:45:44.999202+0000","last_active":"2024-03-16T20:45:45.955744+0000","last_peered":"2024-03-16T20:45:45.955744+0000","last_clean":"2024-03-16T20:45:45.955744+0000","last_became_active":"2024-03-16T20:45:44.997189+0000","last_became_peered":"2024-03-16T20:45:44.997189+0000","last_unstale":"2024-03-16T20:45:45.955744+0000","last_undegraded":"2024-03-16T20:45:45.955744+0000","last_fullsized":"2024-03-16T20:45:45.955744+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:30:32.602019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.266347+0000","last_change":"2024-03-16T20:45:43.947648+0000","last_active":"2024-03-16T20:45:46.266347+0000","last_peered":"2024-03-16T20:45:46.266347+0000","last_clean":"2024-03-16T20:45:46.266347+0000","last_became_active":"2024-03-16T20:45:43.947253+0000","last_became_peered":"2024-03-16T20:45:43.947253+0000","last_unstale":"2024-03-16T20:45:46.266347+0000","last_undegraded":"2024-03-16T20:45:46.266347+0000","last_fullsized":"2024-03-16T20:45:46.266347+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T00:57:50.929258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.268421+0000","last_change":"2024-03-16T20:45:44.987174+0000","last_active":"2024-03-16T20:45:46.268421+0000","last_peered":"2024-03-16T20:45:46.268421+0000","last_clean":"2024-03-16T20:45:46.268421+0000","last_became_active":"2024-03-16T20:45:44.986410+0000","last_became_peered":"2024-03-16T20:45:44.986410+0000","last_unstale":"2024-03-16T20:45:46.268421+0000","last_undegraded":"2024-03-16T20:45:46.268421+0000","last_fullsized":"2024-03-16T20:45:46.268421+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:43.907028+0000","last_clean_scrub_stamp":"2024-03-16T20:45:43.907028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T05:05:57.364887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.267821+0000","last_change":"2024-03-16T20:45:43.942843+0000","last_active":"2024-03-16T20:45:46.267821+0000","last_peered":"2024-03-16T20:45:46.267821+0000","last_clean":"2024-03-16T20:45:46.267821+0000","last_became_active":"2024-03-16T20:45:43.942077+0000","last_became_peered":"2024-03-16T20:45:43.942077+0000","last_unstale":"2024-03-16T20:45:46.267821+0000","last_undegraded":"2024-03-16T20:45:46.267821+0000","last_fullsized":"2024-03-16T20:45:46.267821+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:45:42.898452+0000","last_clean_scrub_stamp":"2024-03-16T20:45:42.898452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T04:39:14.937436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269499+0000","last_change":"2024-03-16T20:44:54.293418+0000","last_active":"2024-03-16T20:45:46.269499+0000","last_peered":"2024-03-16T20:45:46.269499+0000","last_clean":"2024-03-16T20:45:46.269499+0000","last_became_active":"2024-03-16T20:44:54.291347+0000","last_became_peered":"2024-03-16T20:44:54.291347+0000","last_unstale":"2024-03-16T20:45:46.269499+0000","last_undegraded":"2024-03-16T20:45:46.269499+0000","last_fullsized":"2024-03-16T20:45:46.269499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_clean_scrub_stamp":"2024-03-16T20:44:53.077639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:29:18.318494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":933888,"data_stored":719141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":129299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":129355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":128787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":130317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":128753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":352256,"data_stored":130317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343908,"statfs":{"total":96636764160,"available":96608161792,"internally_reserved":0,"allocated":929792,"data_stored":719051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T20:45:53.017 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-16T20:45:53.017 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-03-16T20:45:53.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078f70a640 1 Processor -- start 2024-03-16T20:45:53.191 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078f70a640 1 -- start start 2024-03-16T20:45:53.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078f70a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0788144290 0x7f078806ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:53.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078f70a640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f078806daf0 con 0x7f07880b90c0 2024-03-16T20:45:53.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078f70a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07880b1390 con 0x7f0788144c00 2024-03-16T20:45:53.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078f70a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078806dd40 con 0x7f0788144290 2024-03-16T20:45:53.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078cca5640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f07880b90c0 0x7f078813df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:37430/0 (socket says 172.21.15.122:37430) 2024-03-16T20:45:53.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078cca5640 1 -- 172.21.15.122:0/732539028 learned_addr learned my addr 172.21.15.122:0/732539028 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:53.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078d4a6640 1 --2- 172.21.15.122:0/732539028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0788144290 0x7f078806ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:53.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f077ffff640 1 -- 172.21.15.122:0/732539028 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2654632349 0 0) 0x7f078806daf0 con 0x7f07880b90c0 2024-03-16T20:45:53.195 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f077ffff640 1 -- 172.21.15.122:0/732539028 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0774003530 con 0x7f07880b90c0 2024-03-16T20:45:53.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f077ffff640 1 -- 172.21.15.122:0/732539028 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 930738516 0 0) 0x7f0774003530 con 0x7f07880b90c0 2024-03-16T20:45:53.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f077ffff640 1 -- 172.21.15.122:0/732539028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0788144290 msgr2=0x7f078806ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:53.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f077ffff640 1 --2- 172.21.15.122:0/732539028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0788144290 0x7f078806ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f077ffff640 1 -- 172.21.15.122:0/732539028 >> v1:172.21.15.195:6789/0 conn(0x7f0788144c00 legacy=0x7f078814e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:53.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f077ffff640 1 -- 172.21.15.122:0/732539028 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078813e690 con 0x7f07880b90c0 2024-03-16T20:45:53.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f077ffff640 1 -- 172.21.15.122:0/732539028 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f0778001680 con 0x7f07880b90c0 2024-03-16T20:45:53.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f077ffff640 1 -- 172.21.15.122:0/732539028 >> v1:172.21.15.122:6789/0 conn(0x7f07880b90c0 legacy=0x7f078813df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:53.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f077ffff640 1 --2- 172.21.15.122:0/732539028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0788144290 0x7f07740039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:53.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f077ffff640 1 --2- 172.21.15.122:0/732539028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0774003f30 0x7f07740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:53.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f077ffff640 1 --2- 172.21.15.122:0/732539028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0774004d10 0x7f0774005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:53.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f077ffff640 1 -- 172.21.15.122:0/732539028 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078806dd40 con 0x7f0774004d10 2024-03-16T20:45:53.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f077ffff640 1 -- 172.21.15.122:0/732539028 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077400bbd0 con 0x7f0774003f30 2024-03-16T20:45:53.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f077ffff640 1 -- 172.21.15.122:0/732539028 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077400bd90 con 0x7f0788144290 2024-03-16T20:45:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078dca7640 1 --2- 172.21.15.122:0/732539028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0774004d10 0x7f0774005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078cca5640 1 --2- 172.21.15.122:0/732539028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0774003f30 0x7f07740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078d4a6640 1 --2- 172.21.15.122:0/732539028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0788144290 0x7f07740039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078dca7640 1 -- 172.21.15.122:0/732539028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0788144290 msgr2=0x7f07740039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078dca7640 1 --2- 172.21.15.122:0/732539028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0788144290 0x7f07740039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078dca7640 1 -- 172.21.15.122:0/732539028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0774003f30 msgr2=0x7f07740047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078dca7640 1 --2- 172.21.15.122:0/732539028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0774003f30 0x7f07740047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078dca7640 1 -- 172.21.15.122:0/732539028 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0774003530 con 0x7f0774004d10 2024-03-16T20:45:53.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.187+0000 7f078cca5640 1 --2- 172.21.15.122:0/732539028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0774003f30 0x7f07740047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.191+0000 7f078dca7640 1 --2- 172.21.15.122:0/732539028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0774004d10 0x7f0774005190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0784001e70 tx=0x7f078401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ffffb173d87465 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.191+0000 7f077ffff640 1 -- 172.21.15.122:0/732539028 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784003b80 con 0x7f0774004d10 2024-03-16T20:45:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.191+0000 7f077ffff640 1 -- 172.21.15.122:0/732539028 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0784003ce0 con 0x7f0774004d10 2024-03-16T20:45:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.191+0000 7f077ffff640 1 -- 172.21.15.122:0/732539028 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07840052c0 con 0x7f0774004d10 2024-03-16T20:45:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.191+0000 7f078f70a640 1 -- 172.21.15.122:0/732539028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0774004d10 msgr2=0x7f0774005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.191+0000 7f078f70a640 1 --2- 172.21.15.122:0/732539028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0774004d10 0x7f0774005190 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0784001e70 tx=0x7f078401c780 comp rx=0 tx=0).stop 2024-03-16T20:45:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.191+0000 7f078dca7640 1 -- 172.21.15.122:0/732539028 reap_dead start 2024-03-16T20:45:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.191+0000 7f078f70a640 1 -- 172.21.15.122:0/732539028 shutdown_connections 2024-03-16T20:45:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.191+0000 7f078f70a640 1 -- 172.21.15.122:0/732539028 >> 172.21.15.122:0/732539028 conn(0x7f0788070d30 msgr2=0x7f0788071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:53.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.191+0000 7f078f70a640 1 -- 172.21.15.122:0/732539028 shutdown_connections 2024-03-16T20:45:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.191+0000 7f078f70a640 1 -- 172.21.15.122:0/732539028 wait complete. 2024-03-16T20:45:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.191+0000 7f078f70a640 1 Processor -- start 2024-03-16T20:45:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.191+0000 7f078f70a640 1 -- start start 2024-03-16T20:45:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078f70a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0774004d10 0x7f0788133aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078f70a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0788143a70 0x7f07881ddd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078f70a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07881de250 0x7f07881e1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078f70a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078813d190 con 0x7f0774004d10 2024-03-16T20:45:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078f70a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078813bc60 con 0x7f0788143a70 2024-03-16T20:45:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078f70a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078813bf40 con 0x7f07881de250 2024-03-16T20:45:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078d4a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0774004d10 0x7f0788133aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078d4a6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0774004d10 0x7f0788133aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49178/0 (socket says 172.21.15.122:49178) 2024-03-16T20:45:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078d4a6640 1 -- 172.21.15.122:0/3112793183 learned_addr learned my addr 172.21.15.122:0/3112793183 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078cca5640 1 --2- 172.21.15.122:0/3112793183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0788143a70 0x7f07881ddd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:53.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078dca7640 1 --2- 172.21.15.122:0/3112793183 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07881de250 0x7f07881e1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078d4a6640 1 -- 172.21.15.122:0/3112793183 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07881de250 msgr2=0x7f07881e1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078d4a6640 1 --2- 172.21.15.122:0/3112793183 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07881de250 0x7f07881e1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078d4a6640 1 -- 172.21.15.122:0/3112793183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0788143a70 msgr2=0x7f07881ddd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078d4a6640 1 --2- 172.21.15.122:0/3112793183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0788143a70 0x7f07881ddd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078d4a6640 1 -- 172.21.15.122:0/3112793183 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078813c3d0 con 0x7f0774004d10 2024-03-16T20:45:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078d4a6640 1 --2- 172.21.15.122:0/3112793183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0774004d10 0x7f0788133aa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f07800115e0 tx=0x7f0780011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f077dffb640 1 -- 172.21.15.122:0/3112793183 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078000f8f0 con 0x7f0774004d10 2024-03-16T20:45:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f077dffb640 1 -- 172.21.15.122:0/3112793183 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078000fa50 con 0x7f0774004d10 2024-03-16T20:45:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078f70a640 1 -- 172.21.15.122:0/3112793183 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078813c6b0 con 0x7f0774004d10 2024-03-16T20:45:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f077dffb640 1 -- 172.21.15.122:0/3112793183 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078000fdc0 con 0x7f0774004d10 2024-03-16T20:45:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078f70a640 1 -- 172.21.15.122:0/3112793183 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078813cc70 con 0x7f0774004d10 2024-03-16T20:45:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078f70a640 1 -- 172.21.15.122:0/3112793183 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07880ba6b0 con 0x7f0774004d10 2024-03-16T20:45:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f077dffb640 1 -- 172.21.15.122:0/3112793183 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f0780015460 con 0x7f0774004d10 2024-03-16T20:45:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f077dffb640 1 --2- 172.21.15.122:0/3112793183 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f076c066e40 0x7f076c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:53.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f077dffb640 1 -- 172.21.15.122:0/3112793183 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07800852c0 con 0x7f0774004d10 2024-03-16T20:45:53.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.195+0000 7f078cca5640 1 --2- 172.21.15.122:0/3112793183 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f076c066e40 0x7f076c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:53.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.199+0000 7f078cca5640 1 --2- 172.21.15.122:0/3112793183 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f076c066e40 0x7f076c069300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f07780010f0 tx=0x7f0778009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:53.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.199+0000 7f077dffb640 1 -- 172.21.15.122:0/3112793183 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f0780052140 con 0x7f0774004d10 2024-03-16T20:45:53.420 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.415+0000 7f078f70a640 1 -- 172.21.15.122:0/3112793183 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0788074bb0 con 0x7f0774004d10 2024-03-16T20:45:53.423 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.419+0000 7f077dffb640 1 -- 172.21.15.122:0/3112793183 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f0780055800 con 0x7f0774004d10 2024-03-16T20:45:53.423 INFO:teuthology.orchestra.run.smithi122.stdout:[client.0] 2024-03-16T20:45:53.423 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQCYBPZl3zQpMxAAhyRAHkPMirH6skmDMK55+A== 2024-03-16T20:45:53.424 INFO:teuthology.orchestra.run.smithi122.stdout: caps mds = "allow" 2024-03-16T20:45:53.424 INFO:teuthology.orchestra.run.smithi122.stdout: caps mon = "allow r" 2024-03-16T20:45:53.424 INFO:teuthology.orchestra.run.smithi122.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-03-16T20:45:53.424 INFO:teuthology.orchestra.run.smithi122.stderr:updated caps for client.0 2024-03-16T20:45:53.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.419+0000 7f078f70a640 1 -- 172.21.15.122:0/3112793183 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f076c066e40 msgr2=0x7f076c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:53.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.419+0000 7f078f70a640 1 --2- 172.21.15.122:0/3112793183 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f076c066e40 0x7f076c069300 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f07780010f0 tx=0x7f0778009040 comp rx=0 tx=0).stop 2024-03-16T20:45:53.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.419+0000 7f078f70a640 1 -- 172.21.15.122:0/3112793183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0774004d10 msgr2=0x7f0788133aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:53.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.419+0000 7f078f70a640 1 --2- 172.21.15.122:0/3112793183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0774004d10 0x7f0788133aa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f07800115e0 tx=0x7f0780011aa0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.419+0000 7f078f70a640 1 -- 172.21.15.122:0/3112793183 shutdown_connections 2024-03-16T20:45:53.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.419+0000 7f078f70a640 1 --2- 172.21.15.122:0/3112793183 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f076c066e40 0x7f076c069300 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.419+0000 7f078f70a640 1 --2- 172.21.15.122:0/3112793183 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f07881de250 0x7f07881e1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.419+0000 7f078f70a640 1 --2- 172.21.15.122:0/3112793183 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f0788143a70 0x7f07881ddd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.419+0000 7f078f70a640 1 --2- 172.21.15.122:0/3112793183 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0774004d10 0x7f0788133aa0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.423+0000 7f078f70a640 1 -- 172.21.15.122:0/3112793183 >> 172.21.15.122:0/3112793183 conn(0x7f0788070d30 msgr2=0x7f0788071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:53.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.423+0000 7f078f70a640 1 -- 172.21.15.122:0/3112793183 shutdown_connections 2024-03-16T20:45:53.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.423+0000 7f078f70a640 1 -- 172.21.15.122:0/3112793183 wait complete. 2024-03-16T20:45:53.442 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T20:45:53.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.543+0000 7f33b99d6640 1 Processor -- start 2024-03-16T20:45:53.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.543+0000 7f33b99d6640 1 -- start start 2024-03-16T20:45:53.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.543+0000 7f33b99d6640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f33b41472b0 0x7f33b414a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:53.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.543+0000 7f33b99d6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33b4147af0 0x7f33b40b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:53.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.543+0000 7f33b99d6640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b4072680 con 0x7f33b40b3650 2024-03-16T20:45:53.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.543+0000 7f33b99d6640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b40b45b0 con 0x7f33b41472b0 2024-03-16T20:45:53.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.543+0000 7f33b99d6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40b5760 con 0x7f33b4147af0 2024-03-16T20:45:53.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.543+0000 7f33b2ffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f33b41472b0 0x7f33b414a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:53.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.543+0000 7f33b37fe640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f33b40b3650 0x7f33b406af90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:37440/0 (socket says 172.21.15.122:37440) 2024-03-16T20:45:53.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.543+0000 7f33b37fe640 1 -- 172.21.15.122:0/140616226 learned_addr learned my addr 172.21.15.122:0/140616226 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:53.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.543+0000 7f33b27fc640 1 --2- 172.21.15.122:0/140616226 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33b4147af0 0x7f33b40b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:53.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.543+0000 7f33b2ffd640 1 -- 172.21.15.122:0/140616226 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33b4147af0 msgr2=0x7f33b40b3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:53.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.543+0000 7f33b2ffd640 1 --2- 172.21.15.122:0/140616226 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33b4147af0 0x7f33b40b3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.543+0000 7f33b2ffd640 1 -- 172.21.15.122:0/140616226 >> v1:172.21.15.122:6789/0 conn(0x7f33b40b3650 legacy=0x7f33b406af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:53.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.543+0000 7f33b1ffb640 1 -- 172.21.15.122:0/140616226 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3530605967 0 0) 0x7f33b4072680 con 0x7f33b40b3650 2024-03-16T20:45:53.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.543+0000 7f33b2ffd640 1 -- 172.21.15.122:0/140616226 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b4147f50 con 0x7f33b41472b0 2024-03-16T20:45:53.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.547+0000 7f33b27fc640 1 --2- 172.21.15.122:0/140616226 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33b4147af0 0x7f33b40b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:53.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.547+0000 7f33b2ffd640 1 --2- 172.21.15.122:0/140616226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33b41472b0 0x7f33b414a590 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f33a80097c0 tx=0x7f33a801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6342c95dde86d72 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:53.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.547+0000 7f33b1ffb640 1 -- 172.21.15.122:0/140616226 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a80ab600 con 0x7f33b41472b0 2024-03-16T20:45:53.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.547+0000 7f33b1ffb640 1 -- 172.21.15.122:0/140616226 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a80a7050 con 0x7f33b41472b0 2024-03-16T20:45:53.552 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.547+0000 7f33b1ffb640 1 -- 172.21.15.122:0/140616226 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a80ab980 con 0x7f33b41472b0 2024-03-16T20:45:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.547+0000 7f33b99d6640 1 -- 172.21.15.122:0/140616226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33b41472b0 msgr2=0x7f33b414a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.547+0000 7f33b99d6640 1 --2- 172.21.15.122:0/140616226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33b41472b0 0x7f33b414a590 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f33a80097c0 tx=0x7f33a801c410 comp rx=0 tx=0).stop 2024-03-16T20:45:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.547+0000 7f33b99d6640 1 -- 172.21.15.122:0/140616226 shutdown_connections 2024-03-16T20:45:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.547+0000 7f33b99d6640 1 --2- 172.21.15.122:0/140616226 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33b4147af0 0x7f33b40b3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.547+0000 7f33b99d6640 1 --2- 172.21.15.122:0/140616226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33b41472b0 0x7f33b414a590 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.547+0000 7f33b99d6640 1 -- 172.21.15.122:0/140616226 >> 172.21.15.122:0/140616226 conn(0x7f33b40b62a0 msgr2=0x7f33b40b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.547+0000 7f33b99d6640 1 -- 172.21.15.122:0/140616226 shutdown_connections 2024-03-16T20:45:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.547+0000 7f33b99d6640 1 -- 172.21.15.122:0/140616226 wait complete. 2024-03-16T20:45:53.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b99d6640 1 Processor -- start 2024-03-16T20:45:53.555 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b99d6640 1 -- start start 2024-03-16T20:45:53.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b99d6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33b40b3650 0x7f33b41431c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:53.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b99d6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33b41472b0 0x7f33b41dfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:53.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b99d6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33b4147af0 0x7f33b41dff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:53.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b99d6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41e3310 con 0x7f33b40b3650 2024-03-16T20:45:53.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b99d6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41e3520 con 0x7f33b4147af0 2024-03-16T20:45:53.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b99d6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b41e3800 con 0x7f33b41472b0 2024-03-16T20:45:53.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33b40b3650 0x7f33b41431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:53.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33b40b3650 0x7f33b41431c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49190/0 (socket says 172.21.15.122:49190) 2024-03-16T20:45:53.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b2ffd640 1 -- 172.21.15.122:0/2044687277 learned_addr learned my addr 172.21.15.122:0/2044687277 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:53.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b2ffd640 1 -- 172.21.15.122:0/2044687277 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33b41472b0 msgr2=0x7f33b41dfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:53.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b2ffd640 1 --2- 172.21.15.122:0/2044687277 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33b41472b0 0x7f33b41dfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b2ffd640 1 -- 172.21.15.122:0/2044687277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33b4147af0 msgr2=0x7f33b41dff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:53.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b2ffd640 1 --2- 172.21.15.122:0/2044687277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33b4147af0 0x7f33b41dff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b2ffd640 1 -- 172.21.15.122:0/2044687277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a8009420 con 0x7f33b40b3650 2024-03-16T20:45:53.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b2ffd640 1 --2- 172.21.15.122:0/2044687277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33b40b3650 0x7f33b41431c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f33a80ae040 tx=0x7f33a801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:53.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f339bfff640 1 -- 172.21.15.122:0/2044687277 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a80b7860 con 0x7f33b40b3650 2024-03-16T20:45:53.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f339bfff640 1 -- 172.21.15.122:0/2044687277 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a80b79c0 con 0x7f33b40b3650 2024-03-16T20:45:53.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b99d6640 1 -- 172.21.15.122:0/2044687277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b41e3ae0 con 0x7f33b40b3650 2024-03-16T20:45:53.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f339bfff640 1 -- 172.21.15.122:0/2044687277 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a80b7d70 con 0x7f33b40b3650 2024-03-16T20:45:53.558 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b99d6640 1 -- 172.21.15.122:0/2044687277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b414a590 con 0x7f33b40b3650 2024-03-16T20:45:53.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f33b99d6640 1 -- 172.21.15.122:0/2044687277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b41e3cd0 con 0x7f33b40b3650 2024-03-16T20:45:53.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.551+0000 7f339bfff640 1 -- 172.21.15.122:0/2044687277 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f33a80b5400 con 0x7f33b40b3650 2024-03-16T20:45:53.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.555+0000 7f339bfff640 1 --2- 172.21.15.122:0/2044687277 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f33a0066e40 0x7f33a0069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:53.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.555+0000 7f339bfff640 1 -- 172.21.15.122:0/2044687277 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33a811f590 con 0x7f33b40b3650 2024-03-16T20:45:53.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.555+0000 7f33b27fc640 1 --2- 172.21.15.122:0/2044687277 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f33a0066e40 0x7f33a0069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:53.559 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.555+0000 7f33b27fc640 1 --2- 172.21.15.122:0/2044687277 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f33a0066e40 0x7f33a0069300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f33b41449e0 tx=0x7f33a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:53.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.559+0000 7f339bfff640 1 -- 172.21.15.122:0/2044687277 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f33a80ec410 con 0x7f33b40b3650 2024-03-16T20:45:53.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.747+0000 7f33b99d6640 1 -- 172.21.15.122:0/2044687277 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33b4075280 con 0x7f33b40b3650 2024-03-16T20:45:53.754 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.751+0000 7f339bfff640 1 -- 172.21.15.122:0/2044687277 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+3401 (secure 0 0 0) 0x7f33a80bbd60 con 0x7f33b40b3650 2024-03-16T20:45:53.755 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:53.755 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3638578490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3638578490},{"type":"v1","addr":"172.21.15.195:6835","nonce":3638578490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T20:45:44.280433+0000","modified":"2024-03-16T20:45:46.325382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4995":{"gid":4995,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6833/1841055069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1841055069},{"type":"v1","addr":"172.21.15.122:6833","nonce":1841055069}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4995,"qdb_cluster":[4995]},"id":2}]} 2024-03-16T20:45:53.755 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 13 2024-03-16T20:45:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.751+0000 7f33b99d6640 1 -- 172.21.15.122:0/2044687277 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f33a0066e40 msgr2=0x7f33a0069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.751+0000 7f33b99d6640 1 --2- 172.21.15.122:0/2044687277 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f33a0066e40 0x7f33a0069300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f33b41449e0 tx=0x7f33a4008040 comp rx=0 tx=0).stop 2024-03-16T20:45:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.751+0000 7f33b99d6640 1 -- 172.21.15.122:0/2044687277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33b40b3650 msgr2=0x7f33b41431c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.751+0000 7f33b99d6640 1 --2- 172.21.15.122:0/2044687277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33b40b3650 0x7f33b41431c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f33a80ae040 tx=0x7f33a801ac50 comp rx=0 tx=0).stop 2024-03-16T20:45:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.751+0000 7f33b99d6640 1 -- 172.21.15.122:0/2044687277 shutdown_connections 2024-03-16T20:45:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.751+0000 7f33b99d6640 1 --2- 172.21.15.122:0/2044687277 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f33a0066e40 0x7f33a0069300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.757 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.751+0000 7f33b99d6640 1 --2- 172.21.15.122:0/2044687277 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f33b4147af0 0x7f33b41dff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.751+0000 7f33b99d6640 1 --2- 172.21.15.122:0/2044687277 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f33b41472b0 0x7f33b41dfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.751+0000 7f33b99d6640 1 --2- 172.21.15.122:0/2044687277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f33b40b3650 0x7f33b41431c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:53.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.751+0000 7f33b99d6640 1 -- 172.21.15.122:0/2044687277 >> 172.21.15.122:0/2044687277 conn(0x7f33b40b62a0 msgr2=0x7f33b4071920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:53.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.751+0000 7f33b99d6640 1 -- 172.21.15.122:0/2044687277 shutdown_connections 2024-03-16T20:45:53.758 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:53.751+0000 7f33b99d6640 1 -- 172.21.15.122:0/2044687277 wait complete. 2024-03-16T20:45:53.972 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-03-16T20:45:44.280433+0000', 'modified': '2024-03-16T20:45:46.325382+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4995}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4995': {'gid': 4995, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.122:6833/1841055069', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6832', 'nonce': 1841055069}, {'type': 'v1', 'addr': '172.21.15.122:6833', 'nonce': 1841055069}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4995, 'qdb_cluster': [4995]} 2024-03-16T20:45:53.973 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-03-16T20:45:53.973 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-03-16T20:45:54.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb804c0b640 1 Processor -- start 2024-03-16T20:45:54.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb804c0b640 1 -- start start 2024-03-16T20:45:54.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb804c0b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb800144290 0x7fb80006ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb804c0b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8000b90c0 0x7fb8000b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.090 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb804c0b640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80006daf0 con 0x7fb800144c00 2024-03-16T20:45:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb804c0b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8000b1390 con 0x7fb800144290 2024-03-16T20:45:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb804c0b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80006dd40 con 0x7fb8000b90c0 2024-03-16T20:45:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fed9d640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fb800144c00 0x7fb80013df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:37452/0 (socket says 172.21.15.122:37452) 2024-03-16T20:45:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fed9d640 1 -- 172.21.15.122:0/3195705524 learned_addr learned my addr 172.21.15.122:0/3195705524 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7f5d9b640 1 --2- 172.21.15.122:0/3195705524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8000b90c0 0x7fb8000b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fe59c640 1 --2- 172.21.15.122:0/3195705524 >> v2:172.21.15.195:3300/0 conn(0x7fb800144290 0x7fb80006ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fdd9b640 1 -- 172.21.15.122:0/3195705524 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 785905207 0 0) 0x7fb80006daf0 con 0x7fb800144c00 2024-03-16T20:45:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fdd9b640 1 -- 172.21.15.122:0/3195705524 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7ec003530 con 0x7fb800144c00 2024-03-16T20:45:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fdd9b640 1 -- 172.21.15.122:0/3195705524 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3561829074 0 0) 0x7fb7ec003530 con 0x7fb800144c00 2024-03-16T20:45:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fdd9b640 1 -- 172.21.15.122:0/3195705524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8000b90c0 msgr2=0x7fb8000b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fdd9b640 1 --2- 172.21.15.122:0/3195705524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8000b90c0 0x7fb8000b7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fdd9b640 1 -- 172.21.15.122:0/3195705524 >> v2:172.21.15.195:3300/0 conn(0x7fb800144290 msgr2=0x7fb80006ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fdd9b640 1 --2- 172.21.15.122:0/3195705524 >> v2:172.21.15.195:3300/0 conn(0x7fb800144290 0x7fb80006ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fdd9b640 1 -- 172.21.15.122:0/3195705524 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80013e690 con 0x7fb800144c00 2024-03-16T20:45:54.091 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fdd9b640 1 -- 172.21.15.122:0/3195705524 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7fb7f8001a00 con 0x7fb800144c00 2024-03-16T20:45:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fdd9b640 1 -- 172.21.15.122:0/3195705524 >> v1:172.21.15.122:6789/0 conn(0x7fb800144c00 legacy=0x7fb80013df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fdd9b640 1 --2- 172.21.15.122:0/3195705524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8000b90c0 0x7fb7ec003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fdd9b640 1 --2- 172.21.15.122:0/3195705524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7ec003f00 0x7fb7ec0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fdd9b640 1 --2- 172.21.15.122:0/3195705524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7ec004d10 0x7fb7ec005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fdd9b640 1 -- 172.21.15.122:0/3195705524 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8000b1390 con 0x7fb7ec003f00 2024-03-16T20:45:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fdd9b640 1 -- 172.21.15.122:0/3195705524 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80006dd40 con 0x7fb7ec004d10 2024-03-16T20:45:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fdd9b640 1 -- 172.21.15.122:0/3195705524 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7ec00ba70 con 0x7fb8000b90c0 2024-03-16T20:45:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7f5d9b640 1 --2- 172.21.15.122:0/3195705524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7ec003f00 0x7fb7ec0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fe59c640 1 --2- 172.21.15.122:0/3195705524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8000b90c0 0x7fb7ec003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7fed9d640 1 --2- 172.21.15.122:0/3195705524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7ec004d10 0x7fb7ec005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7f5d9b640 1 -- 172.21.15.122:0/3195705524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8000b90c0 msgr2=0x7fb7ec003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:54.092 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7f5d9b640 1 --2- 172.21.15.122:0/3195705524 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8000b90c0 0x7fb7ec003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7f5d9b640 1 -- 172.21.15.122:0/3195705524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7ec004d10 msgr2=0x7fb7ec005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7f5d9b640 1 --2- 172.21.15.122:0/3195705524 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb7ec004d10 0x7fb7ec005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.083+0000 7fb7f5d9b640 1 -- 172.21.15.122:0/3195705524 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec003530 con 0x7fb7ec003f00 2024-03-16T20:45:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.087+0000 7fb7f5d9b640 1 --2- 172.21.15.122:0/3195705524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7ec003f00 0x7fb7ec0047d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009ab0 tx=0x7fb7e801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d9b1f99901dcac server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.087+0000 7fb7fdd9b640 1 -- 172.21.15.122:0/3195705524 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e80a5640 con 0x7fb7ec003f00 2024-03-16T20:45:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.087+0000 7fb7fdd9b640 1 -- 172.21.15.122:0/3195705524 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e80a57a0 con 0x7fb7ec003f00 2024-03-16T20:45:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.087+0000 7fb7fdd9b640 1 -- 172.21.15.122:0/3195705524 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e80a5b50 con 0x7fb7ec003f00 2024-03-16T20:45:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.087+0000 7fb804c0b640 1 -- 172.21.15.122:0/3195705524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7ec003f00 msgr2=0x7fb7ec0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.087+0000 7fb804c0b640 1 --2- 172.21.15.122:0/3195705524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7ec003f00 0x7fb7ec0047d0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8009ab0 tx=0x7fb7e801ca50 comp rx=0 tx=0).stop 2024-03-16T20:45:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.087+0000 7fb7fed9d640 1 -- 172.21.15.122:0/3195705524 reap_dead start 2024-03-16T20:45:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.087+0000 7fb804c0b640 1 -- 172.21.15.122:0/3195705524 shutdown_connections 2024-03-16T20:45:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.087+0000 7fb804c0b640 1 -- 172.21.15.122:0/3195705524 >> 172.21.15.122:0/3195705524 conn(0x7fb800070d30 msgr2=0x7fb800071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.087+0000 7fb804c0b640 1 -- 172.21.15.122:0/3195705524 shutdown_connections 2024-03-16T20:45:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.087+0000 7fb804c0b640 1 -- 172.21.15.122:0/3195705524 wait complete. 2024-03-16T20:45:54.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.087+0000 7fb804c0b640 1 Processor -- start 2024-03-16T20:45:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.087+0000 7fb804c0b640 1 -- start start 2024-03-16T20:45:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb804c0b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7ec003f00 0x7fb8001379d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb804c0b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8001d5310 0x7fb800137f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb804c0b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8000b2540 0x7fb8001e3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb804c0b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8001d4470 con 0x7fb7ec003f00 2024-03-16T20:45:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb804c0b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8001d57f0 con 0x7fb8000b2540 2024-03-16T20:45:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb804c0b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8001d5ad0 con 0x7fb8001d5310 2024-03-16T20:45:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7fe59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7ec003f00 0x7fb8001379d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7fe59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7ec003f00 0x7fb8001379d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49210/0 (socket says 172.21.15.122:49210) 2024-03-16T20:45:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7fe59c640 1 -- 172.21.15.122:0/1698249281 learned_addr learned my addr 172.21.15.122:0/1698249281 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7fed9d640 1 --2- 172.21.15.122:0/1698249281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8000b2540 0x7fb8001e3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7f5d9b640 1 --2- 172.21.15.122:0/1698249281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8001d5310 0x7fb800137f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.097 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7fe59c640 1 -- 172.21.15.122:0/1698249281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8001d5310 msgr2=0x7fb800137f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:54.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7fe59c640 1 --2- 172.21.15.122:0/1698249281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8001d5310 0x7fb800137f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7fe59c640 1 -- 172.21.15.122:0/1698249281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8000b2540 msgr2=0x7fb8001e3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:54.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7fe59c640 1 --2- 172.21.15.122:0/1698249281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8000b2540 0x7fb8001e3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7fe59c640 1 -- 172.21.15.122:0/1698249281 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8000b3500 con 0x7fb7ec003f00 2024-03-16T20:45:54.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7fe59c640 1 --2- 172.21.15.122:0/1698249281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7ec003f00 0x7fb8001379d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0011660 tx=0x7fb7f0011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:54.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7f77fe640 1 -- 172.21.15.122:0/1698249281 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f000f9c0 con 0x7fb7ec003f00 2024-03-16T20:45:54.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7f77fe640 1 -- 172.21.15.122:0/1698249281 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f000fb20 con 0x7fb7ec003f00 2024-03-16T20:45:54.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7f77fe640 1 -- 172.21.15.122:0/1698249281 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f000fe90 con 0x7fb7ec003f00 2024-03-16T20:45:54.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb804c0b640 1 -- 172.21.15.122:0/1698249281 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8000b37e0 con 0x7fb7ec003f00 2024-03-16T20:45:54.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb804c0b640 1 -- 172.21.15.122:0/1698249281 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8001e37c0 con 0x7fb7ec003f00 2024-03-16T20:45:54.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7f77fe640 1 -- 172.21.15.122:0/1698249281 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fb7f0015460 con 0x7fb7ec003f00 2024-03-16T20:45:54.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7f77fe640 1 --2- 172.21.15.122:0/1698249281 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb7e0066e40 0x7fb7e0069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7f77fe640 1 -- 172.21.15.122:0/1698249281 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7f0085320 con 0x7fb7ec003f00 2024-03-16T20:45:54.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb7f5d9b640 1 --2- 172.21.15.122:0/1698249281 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb7e0066e40 0x7fb7e0069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb807d7d000 1 Processor -- start 2024-03-16T20:45:54.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.091+0000 7fb807d7d000 1 -- start start 2024-03-16T20:45:54.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7f5d9b640 1 --2- 172.21.15.122:0/1698249281 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb7e0066e40 0x7fb7e0069300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80af000 tx=0x7fb7e80046d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:54.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57ddd0 0x55da3f57e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57e6f0 0x55da3f57ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57f010 0x55da3f5b8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3f445d00 con 0x55da3f57f010 2024-03-16T20:45:54.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3f3d7430 con 0x55da3f57e6f0 2024-03-16T20:45:54.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da3f3d4960 con 0x55da3f57ddd0 2024-03-16T20:45:54.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57f010 0x55da3f5b8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fed9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57f010 0x55da3f5b8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49224/0 (socket says 172.21.15.122:49224) 2024-03-16T20:45:54.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fed9d640 1 -- 172.21.15.122:0/61980445 learned_addr learned my addr 172.21.15.122:0/61980445 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:54.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fe59c640 1 --2- 172.21.15.122:0/61980445 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57ddd0 0x55da3f57e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7f5d9b640 1 --2- 172.21.15.122:0/61980445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57e6f0 0x55da3f57ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fed9d640 1 -- 172.21.15.122:0/61980445 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57ddd0 msgr2=0x55da3f57e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:54.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fed9d640 1 --2- 172.21.15.122:0/61980445 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57ddd0 0x55da3f57e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fed9d640 1 -- 172.21.15.122:0/61980445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57e6f0 msgr2=0x55da3f57ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:54.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fed9d640 1 --2- 172.21.15.122:0/61980445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57e6f0 0x55da3f57ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fed9d640 1 -- 172.21.15.122:0/61980445 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da3f0c6890 con 0x55da3f57f010 2024-03-16T20:45:54.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fe59c640 1 --2- 172.21.15.122:0/61980445 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57ddd0 0x55da3f57e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:54.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7f5d9b640 1 --2- 172.21.15.122:0/61980445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57e6f0 0x55da3f57ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:54.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fed9d640 1 --2- 172.21.15.122:0/61980445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57f010 0x55da3f5b8dc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8002080 tx=0x7fb7f8005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840e351f714ccc9b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:54.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7dffff640 1 -- 172.21.15.122:0/61980445 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8007780 con 0x55da3f57f010 2024-03-16T20:45:54.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7dffff640 1 -- 172.21.15.122:0/61980445 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f80078e0 con 0x55da3f57f010 2024-03-16T20:45:54.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7dffff640 1 -- 172.21.15.122:0/61980445 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8014400 con 0x55da3f57f010 2024-03-16T20:45:54.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 -- 172.21.15.122:0/61980445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57f010 msgr2=0x55da3f5b8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:54.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 --2- 172.21.15.122:0/61980445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57f010 0x55da3f5b8dc0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8002080 tx=0x7fb7f8005f50 comp rx=0 tx=0).stop 2024-03-16T20:45:54.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 -- 172.21.15.122:0/61980445 shutdown_connections 2024-03-16T20:45:54.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 --2- 172.21.15.122:0/61980445 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57f010 0x55da3f5b8dc0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 --2- 172.21.15.122:0/61980445 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57e6f0 0x55da3f57ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 --2- 172.21.15.122:0/61980445 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57ddd0 0x55da3f57e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 -- 172.21.15.122:0/61980445 >> 172.21.15.122:0/61980445 conn(0x55da3f57a2c0 msgr2=0x55da3f57c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:54.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 -- 172.21.15.122:0/61980445 shutdown_connections 2024-03-16T20:45:54.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 -- 172.21.15.122:0/61980445 wait complete. 2024-03-16T20:45:54.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 -- start start 2024-03-16T20:45:54.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 Processor -- start 2024-03-16T20:45:54.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57ddd0 0x55da3f5b9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57e6f0 0x55da3f5b9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57f010 0x55da3f589b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3f3d0140 con 0x55da3f57ddd0 2024-03-16T20:45:54.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3f36a630 con 0x55da3f57e6f0 2024-03-16T20:45:54.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da3f36a2d0 con 0x55da3f57f010 2024-03-16T20:45:54.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb807d7d000 10 client.0 _ll_register_callbacks cb 0x55da3f4801c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T20:45:54.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fe59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57ddd0 0x55da3f5b9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fe59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57ddd0 0x55da3f5b9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49234/0 (socket says 172.21.15.122:49234) 2024-03-16T20:45:54.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fe59c640 1 -- 172.21.15.122:0/1246366458 learned_addr learned my addr 172.21.15.122:0/1246366458 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:54.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7f5d9b640 1 --2- 172.21.15.122:0/1246366458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57e6f0 0x55da3f5b9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fe59c640 1 -- 172.21.15.122:0/1246366458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57f010 msgr2=0x55da3f589b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:54.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fe59c640 1 --2- 172.21.15.122:0/1246366458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57f010 0x55da3f589b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fe59c640 1 -- 172.21.15.122:0/1246366458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57e6f0 msgr2=0x55da3f5b9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:54.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fe59c640 1 --2- 172.21.15.122:0/1246366458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57e6f0 0x55da3f5b9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fe59c640 1 -- 172.21.15.122:0/1246366458 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f800f040 con 0x55da3f57ddd0 2024-03-16T20:45:54.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7f5d9b640 1 --2- 172.21.15.122:0/1246366458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57e6f0 0x55da3f5b9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:54.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.095+0000 7fb7fe59c640 1 --2- 172.21.15.122:0/1246366458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57ddd0 0x55da3f5b9a80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0052410 tx=0x7fb7f0055a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2115d44fbfe1af server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:54.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb7dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-16T20:45:54.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb7dd7fa640 1 -- 172.21.15.122:0/1246366458 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f005c0e0 con 0x55da3f57ddd0 2024-03-16T20:45:54.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb7dd7fa640 1 -- 172.21.15.122:0/1246366458 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7f005c240 con 0x55da3f57ddd0 2024-03-16T20:45:54.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 1 -- 172.21.15.122:0/1246366458 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da3f383850 con 0x55da3f57ddd0 2024-03-16T20:45:54.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb7dd7fa640 1 -- 172.21.15.122:0/1246366458 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7f0055400 con 0x55da3f57ddd0 2024-03-16T20:45:54.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 10 client.5199 fetch_fsmap learned FSMap version 13 2024-03-16T20:45:54.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 1 -- 172.21.15.122:0/1246366458 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55da3f38fe70 con 0x55da3f57ddd0 2024-03-16T20:45:54.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb7dd7fa640 1 -- 172.21.15.122:0/1246366458 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55da3f383850 con 0x55da3f57ddd0 2024-03-16T20:45:54.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb7dd7fa640 1 -- 172.21.15.122:0/1246366458 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 13) v1 ==== 2164+0+0 (secure 0 0 0) 0x7fb7f001b020 con 0x55da3f57ddd0 2024-03-16T20:45:54.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 10 client.5199 fetch_fsmap finished waiting for FSMap version 13 2024-03-16T20:45:54.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 10 client.5199 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-16T20:45:54.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 20 client.5199 populate_metadata read hostname 'smithi122' 2024-03-16T20:45:54.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 1 --2- 172.21.15.122:0/1246366458 >> [v2:172.21.15.122:6832/1841055069,v1:172.21.15.122:6833/1841055069] conn(0x55da3f58a8e0 0x55da3f58acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 4 client.5199 mds_command: new command op to 4995 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-16T20:45:54.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 1 -- 172.21.15.122:0/1246366458 --> [v2:172.21.15.122:6832/1841055069,v1:172.21.15.122:6833/1841055069] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da3f3ccf20 con 0x55da3f58a8e0 2024-03-16T20:45:54.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb7f5d9b640 1 --2- 172.21.15.122:0/1246366458 >> [v2:172.21.15.122:6832/1841055069,v1:172.21.15.122:6833/1841055069] conn(0x55da3f58a8e0 0x55da3f58acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb7dd7fa640 10 client.5199 ms_handle_connect on v2:172.21.15.122:6832/1841055069 2024-03-16T20:45:54.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb7f5d9b640 1 --2- 172.21.15.122:0/1246366458 >> [v2:172.21.15.122:6832/1841055069,v1:172.21.15.122:6833/1841055069] conn(0x55da3f58a8e0 0x55da3f58acc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3009426c1fab2bca server_cookie=f342a877ed7de659 in_seq=0 out_seq=0 2024-03-16T20:45:54.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb7dd7fa640 1 -- 172.21.15.122:0/1246366458 <== mds.0 v2:172.21.15.122:6832/1841055069 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55da3f3ccf20 con 0x55da3f58a8e0 2024-03-16T20:45:54.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb7dd7fa640 10 client.5199 handle_command_reply: tid=0 2024-03-16T20:45:54.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 1 client.5199 shutdown 2024-03-16T20:45:54.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 1 -- 172.21.15.122:0/1246366458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57ddd0 msgr2=0x55da3f5b9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:54.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 1 --2- 172.21.15.122:0/1246366458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57ddd0 0x55da3f5b9a80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0052410 tx=0x7fb7f0055a00 comp rx=0 tx=0).stop 2024-03-16T20:45:54.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 1 -- 172.21.15.122:0/1246366458 shutdown_connections 2024-03-16T20:45:54.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 1 --2- 172.21.15.122:0/1246366458 >> [v2:172.21.15.122:6832/1841055069,v1:172.21.15.122:6833/1841055069] conn(0x55da3f58a8e0 0x55da3f58acc0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.112 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 1 --2- 172.21.15.122:0/1246366458 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57f010 0x55da3f589b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 1 --2- 172.21.15.122:0/1246366458 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57e6f0 0x55da3f5b9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb7dd7fa640 0 client.5199 ms_handle_reset on v2:172.21.15.122:6832/1841055069 2024-03-16T20:45:54.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 1 --2- 172.21.15.122:0/1246366458 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57ddd0 0x55da3f5b9a80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 1 -- 172.21.15.122:0/1246366458 >> 172.21.15.122:0/1246366458 conn(0x55da3f57a2c0 msgr2=0x55da3f57cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:54.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 1 -- 172.21.15.122:0/1246366458 shutdown_connections 2024-03-16T20:45:54.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 1 -- 172.21.15.122:0/1246366458 wait complete. 2024-03-16T20:45:54.113 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.099+0000 7fb807d7d000 20 client.5199 trim_cache size 0 max 16384 2024-03-16T20:45:54.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 Processor -- start 2024-03-16T20:45:54.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 -- start start 2024-03-16T20:45:54.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57ddd0 0x55da3f5c0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57e6f0 0x55da3f5bb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57f010 0x55da3f5bbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3f589f40 con 0x55da3f57ddd0 2024-03-16T20:45:54.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3f087940 con 0x55da3f57f010 2024-03-16T20:45:54.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da3f392a60 con 0x55da3f57e6f0 2024-03-16T20:45:54.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb7fe59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57ddd0 0x55da3f5c0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb7fe59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57ddd0 0x55da3f5c0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49238/0 (socket says 172.21.15.122:49238) 2024-03-16T20:45:54.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb7fe59c640 1 -- 172.21.15.122:0/1055913073 learned_addr learned my addr 172.21.15.122:0/1055913073 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:54.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb7fe59c640 1 -- 172.21.15.122:0/1055913073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57e6f0 msgr2=0x55da3f5bb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:54.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb7fe59c640 1 --2- 172.21.15.122:0/1055913073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57e6f0 0x55da3f5bb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb7fe59c640 1 -- 172.21.15.122:0/1055913073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57f010 msgr2=0x55da3f5bbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:54.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb7fe59c640 1 --2- 172.21.15.122:0/1055913073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57f010 0x55da3f5bbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb7fe59c640 1 -- 172.21.15.122:0/1055913073 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da3f375e10 con 0x55da3f57ddd0 2024-03-16T20:45:54.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb7fe59c640 1 --2- 172.21.15.122:0/1055913073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57ddd0 0x55da3f5c0150 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00523a0 tx=0x7fb7f0085050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c57395f339e8e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:54.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb7df7fe640 1 -- 172.21.15.122:0/1055913073 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f005a050 con 0x55da3f57ddd0 2024-03-16T20:45:54.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb7df7fe640 1 -- 172.21.15.122:0/1055913073 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7f0054050 con 0x55da3f57ddd0 2024-03-16T20:45:54.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb7df7fe640 1 -- 172.21.15.122:0/1055913073 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7f00542b0 con 0x55da3f57ddd0 2024-03-16T20:45:54.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 -- 172.21.15.122:0/1055913073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57ddd0 msgr2=0x55da3f5c0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:54.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 --2- 172.21.15.122:0/1055913073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57ddd0 0x55da3f5c0150 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb7f00523a0 tx=0x7fb7f0085050 comp rx=0 tx=0).stop 2024-03-16T20:45:54.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 -- 172.21.15.122:0/1055913073 shutdown_connections 2024-03-16T20:45:54.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 --2- 172.21.15.122:0/1055913073 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57f010 0x55da3f5bbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 --2- 172.21.15.122:0/1055913073 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57e6f0 0x55da3f5bb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 --2- 172.21.15.122:0/1055913073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57ddd0 0x55da3f5c0150 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 -- 172.21.15.122:0/1055913073 >> 172.21.15.122:0/1055913073 conn(0x55da3f57a2c0 msgr2=0x55da3f57c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:54.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 -- 172.21.15.122:0/1055913073 shutdown_connections 2024-03-16T20:45:54.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 -- 172.21.15.122:0/1055913073 wait complete. 2024-03-16T20:45:54.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 -- start start 2024-03-16T20:45:54.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.115+0000 7fb807d7d000 1 Processor -- start 2024-03-16T20:45:54.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57ddd0 0x55da3f614ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57e6f0 0x55da3f615220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57f010 0x55da3f5bec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3f3cd540 con 0x55da3f57e6f0 2024-03-16T20:45:54.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da3f3ca2c0 con 0x55da3f57f010 2024-03-16T20:45:54.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da3f388800 con 0x55da3f57ddd0 2024-03-16T20:45:54.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 10 client.0 _ll_register_callbacks cb 0x7fb7f0055b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-16T20:45:54.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7f5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57e6f0 0x55da3f615220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7f5d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57e6f0 0x55da3f615220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49248/0 (socket says 172.21.15.122:49248) 2024-03-16T20:45:54.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7f5d9b640 1 -- 172.21.15.122:0/3102846271 learned_addr learned my addr 172.21.15.122:0/3102846271 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:54.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7fe59c640 1 --2- 172.21.15.122:0/3102846271 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57ddd0 0x55da3f614ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7fed9d640 1 --2- 172.21.15.122:0/3102846271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57f010 0x55da3f5bec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7f5d9b640 1 -- 172.21.15.122:0/3102846271 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57ddd0 msgr2=0x55da3f614ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:54.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7f5d9b640 1 --2- 172.21.15.122:0/3102846271 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57ddd0 0x55da3f614ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7f5d9b640 1 -- 172.21.15.122:0/3102846271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57f010 msgr2=0x55da3f5bec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:54.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7f5d9b640 1 --2- 172.21.15.122:0/3102846271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57f010 0x55da3f5bec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7f5d9b640 1 -- 172.21.15.122:0/3102846271 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da3f38fe70 con 0x55da3f57e6f0 2024-03-16T20:45:54.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7fed9d640 1 --2- 172.21.15.122:0/3102846271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57f010 0x55da3f5bec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:54.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7fe59c640 1 --2- 172.21.15.122:0/3102846271 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57ddd0 0x55da3f614ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:54.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7f5d9b640 1 --2- 172.21.15.122:0/3102846271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57e6f0 0x55da3f615220 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8000c10 tx=0x7fb7ec005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2edf5038529acd36 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:54.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-03-16T20:45:54.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7dd7fa640 1 -- 172.21.15.122:0/3102846271 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e80a5b10 con 0x55da3f57e6f0 2024-03-16T20:45:54.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7dd7fa640 1 -- 172.21.15.122:0/3102846271 <== mon.0 v2:172.21.15.122:3300/0 2 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7e80a5c70 con 0x55da3f57e6f0 2024-03-16T20:45:54.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 -- 172.21.15.122:0/3102846271 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da3f589a00 con 0x55da3f57e6f0 2024-03-16T20:45:54.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7dd7fa640 1 -- 172.21.15.122:0/3102846271 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7e80a5e30 con 0x55da3f57e6f0 2024-03-16T20:45:54.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 10 client.5205 fetch_fsmap learned FSMap version 13 2024-03-16T20:45:54.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 -- 172.21.15.122:0/3102846271 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x55da3f0c6cc0 con 0x55da3f57e6f0 2024-03-16T20:45:54.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7dd7fa640 1 -- 172.21.15.122:0/3102846271 <== mon.0 v2:172.21.15.122:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55da3f589a00 con 0x55da3f57e6f0 2024-03-16T20:45:54.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7dd7fa640 1 -- 172.21.15.122:0/3102846271 <== mon.0 v2:172.21.15.122:3300/0 5 ==== fsmap(e 13) v1 ==== 2164+0+0 (secure 0 0 0) 0x7fb7e8003620 con 0x55da3f57e6f0 2024-03-16T20:45:54.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 10 client.5205 fetch_fsmap finished waiting for FSMap version 13 2024-03-16T20:45:54.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 10 client.5205 resolve_mds: resolved name 'a' to daemon mds.a 2024-03-16T20:45:54.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 20 client.5205 populate_metadata read hostname 'smithi122' 2024-03-16T20:45:54.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 --2- 172.21.15.122:0/3102846271 >> [v2:172.21.15.122:6832/1841055069,v1:172.21.15.122:6833/1841055069] conn(0x55da3f5bff80 0x55da3f620d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 4 client.5205 mds_command: new command op to 4995 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-16T20:45:54.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 -- 172.21.15.122:0/3102846271 --> [v2:172.21.15.122:6832/1841055069,v1:172.21.15.122:6833/1841055069] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55da3f355e80 con 0x55da3f5bff80 2024-03-16T20:45:54.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7fe59c640 1 --2- 172.21.15.122:0/3102846271 >> [v2:172.21.15.122:6832/1841055069,v1:172.21.15.122:6833/1841055069] conn(0x55da3f5bff80 0x55da3f620d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7dd7fa640 10 client.5205 ms_handle_connect on v2:172.21.15.122:6832/1841055069 2024-03-16T20:45:54.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7fe59c640 1 --2- 172.21.15.122:0/3102846271 >> [v2:172.21.15.122:6832/1841055069,v1:172.21.15.122:6833/1841055069] conn(0x55da3f5bff80 0x55da3f620d90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34defca3e3bdacf0 server_cookie=f63ab484375bca82 in_seq=0 out_seq=0 2024-03-16T20:45:54.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7dd7fa640 1 -- 172.21.15.122:0/3102846271 <== mds.0 v2:172.21.15.122:6832/1841055069 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55da3f355e80 con 0x55da3f5bff80 2024-03-16T20:45:54.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7dd7fa640 10 client.5205 handle_command_reply: tid=0 2024-03-16T20:45:54.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 client.5205 shutdown 2024-03-16T20:45:54.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 -- 172.21.15.122:0/3102846271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57e6f0 msgr2=0x55da3f615220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:54.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 --2- 172.21.15.122:0/3102846271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57e6f0 0x55da3f615220 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb7e8000c10 tx=0x7fb7ec005800 comp rx=0 tx=0).stop 2024-03-16T20:45:54.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 -- 172.21.15.122:0/3102846271 shutdown_connections 2024-03-16T20:45:54.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 --2- 172.21.15.122:0/3102846271 >> [v2:172.21.15.122:6832/1841055069,v1:172.21.15.122:6833/1841055069] conn(0x55da3f5bff80 0x55da3f620d90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 --2- 172.21.15.122:0/3102846271 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x55da3f57f010 0x55da3f5bec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 --2- 172.21.15.122:0/3102846271 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55da3f57e6f0 0x55da3f615220 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb7dd7fa640 0 client.5205 ms_handle_reset on v2:172.21.15.122:6832/1841055069 2024-03-16T20:45:54.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 --2- 172.21.15.122:0/3102846271 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x55da3f57ddd0 0x55da3f614ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 -- 172.21.15.122:0/3102846271 >> 172.21.15.122:0/3102846271 conn(0x55da3f57a2c0 msgr2=0x55da3f5c9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:54.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 -- 172.21.15.122:0/3102846271 shutdown_connections 2024-03-16T20:45:54.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 1 -- 172.21.15.122:0/3102846271 wait complete. 2024-03-16T20:45:54.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb807d7d000 20 client.5205 trim_cache size 0 max 16384 2024-03-16T20:45:54.132 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-16T20:45:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "cluster_fsid": "ea4c964c-3481-413d-a556-a3fc2950fdd6", 2024-03-16T20:45:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "whoami": 0, 2024-03-16T20:45:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 4995, 2024-03-16T20:45:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "want_state": "up:active", 2024-03-16T20:45:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "up:active", 2024-03-16T20:45:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs", 2024-03-16T20:45:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "rank_uptime": 9.1288715039999992, 2024-03-16T20:45:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "mdsmap_epoch": 13, 2024-03-16T20:45:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch": 30, 2024-03-16T20:45:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_epoch_barrier": 30, 2024-03-16T20:45:54.133 INFO:teuthology.orchestra.run.smithi122.stdout: "uptime": 16.091979846000001 2024-03-16T20:45:54.133 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-16T20:45:54.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb804c0b640 1 -- 172.21.15.122:0/1698249281 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb7e0066e40 msgr2=0x7fb7e0069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:54.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb804c0b640 1 --2- 172.21.15.122:0/1698249281 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb7e0066e40 0x7fb7e0069300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb7e80af000 tx=0x7fb7e80046d0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb804c0b640 1 -- 172.21.15.122:0/1698249281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7ec003f00 msgr2=0x7fb8001379d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:54.134 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb804c0b640 1 --2- 172.21.15.122:0/1698249281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7ec003f00 0x7fb8001379d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0011660 tx=0x7fb7f0011b20 comp rx=0 tx=0).stop 2024-03-16T20:45:54.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb804c0b640 1 -- 172.21.15.122:0/1698249281 shutdown_connections 2024-03-16T20:45:54.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb804c0b640 1 --2- 172.21.15.122:0/1698249281 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb7e0066e40 0x7fb7e0069300 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb804c0b640 1 --2- 172.21.15.122:0/1698249281 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb8000b2540 0x7fb8001e3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb804c0b640 1 --2- 172.21.15.122:0/1698249281 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8001d5310 0x7fb800137f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb804c0b640 1 --2- 172.21.15.122:0/1698249281 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb7ec003f00 0x7fb8001379d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb804c0b640 1 -- 172.21.15.122:0/1698249281 >> 172.21.15.122:0/1698249281 conn(0x7fb800070d30 msgr2=0x7fb80014cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:54.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb804c0b640 1 -- 172.21.15.122:0/1698249281 shutdown_connections 2024-03-16T20:45:54.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.119+0000 7fb804c0b640 1 -- 172.21.15.122:0/1698249281 wait complete. 2024-03-16T20:45:54.140 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-03-16T20:45:54.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.239+0000 7f62dfac7640 1 Processor -- start 2024-03-16T20:45:54.244 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.239+0000 7f62dfac7640 1 -- start start 2024-03-16T20:45:54.245 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.239+0000 7f62dfac7640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f62d8148bd0 0x7f62d813ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.239+0000 7f62dfac7640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f62d8071df0 0x7f62d813f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.239+0000 7f62dfac7640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d80b39f0 con 0x7f62d8071a10 2024-03-16T20:45:54.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.239+0000 7f62dfac7640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d813f750 con 0x7f62d8148bd0 2024-03-16T20:45:54.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.239+0000 7f62dfac7640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d80b3680 con 0x7f62d8071df0 2024-03-16T20:45:54.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.239+0000 7f62dd062640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f62d8148bd0 0x7f62d813ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.239+0000 7f62dd062640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f62d8148bd0 0x7f62d813ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49250/0 (socket says 172.21.15.122:49250) 2024-03-16T20:45:54.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.239+0000 7f62dd062640 1 -- 172.21.15.122:0/1978961745 learned_addr learned my addr 172.21.15.122:0/1978961745 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:54.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.239+0000 7f62dd863640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f62d8071a10 0x7f62d81484c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:37456/0 (socket says 172.21.15.122:37456) 2024-03-16T20:45:54.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.239+0000 7f62de064640 1 --2- 172.21.15.122:0/1978961745 >> v2:172.21.15.195:3300/0 conn(0x7f62d8071df0 0x7f62d813f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.246 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.239+0000 7f62dd062640 1 -- 172.21.15.122:0/1978961745 >> v2:172.21.15.195:3300/0 conn(0x7f62d8071df0 msgr2=0x7f62d813f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:54.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.239+0000 7f62dd062640 1 --2- 172.21.15.122:0/1978961745 >> v2:172.21.15.195:3300/0 conn(0x7f62d8071df0 0x7f62d813f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.239+0000 7f62dd062640 1 -- 172.21.15.122:0/1978961745 >> v1:172.21.15.122:6789/0 conn(0x7f62d8071a10 legacy=0x7f62d81484c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:54.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.239+0000 7f62dd062640 1 -- 172.21.15.122:0/1978961745 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d80b3390 con 0x7f62d8148bd0 2024-03-16T20:45:54.247 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.239+0000 7f62dc861640 1 -- 172.21.15.122:0/1978961745 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067729598 0 0) 0x7f62d80b39f0 con 0x7f62d8071a10 2024-03-16T20:45:54.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dd062640 1 --2- 172.21.15.122:0/1978961745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8148bd0 0x7f62d813ecd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f62d8146060 tx=0x7f62c801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e97504ce5d6570 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:54.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dc861640 1 -- 172.21.15.122:0/1978961745 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c80a9400 con 0x7f62d8148bd0 2024-03-16T20:45:54.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dc861640 1 -- 172.21.15.122:0/1978961745 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c80a9560 con 0x7f62d8148bd0 2024-03-16T20:45:54.248 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dc861640 1 -- 172.21.15.122:0/1978961745 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c80a98f0 con 0x7f62d8148bd0 2024-03-16T20:45:54.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dfac7640 1 -- 172.21.15.122:0/1978961745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8148bd0 msgr2=0x7f62d813ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:54.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dfac7640 1 --2- 172.21.15.122:0/1978961745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8148bd0 0x7f62d813ecd0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f62d8146060 tx=0x7f62c801c520 comp rx=0 tx=0).stop 2024-03-16T20:45:54.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dfac7640 1 -- 172.21.15.122:0/1978961745 shutdown_connections 2024-03-16T20:45:54.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dfac7640 1 --2- 172.21.15.122:0/1978961745 >> v2:172.21.15.195:3300/0 conn(0x7f62d8071df0 0x7f62d813f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dfac7640 1 --2- 172.21.15.122:0/1978961745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8148bd0 0x7f62d813ecd0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dfac7640 1 -- 172.21.15.122:0/1978961745 >> 172.21.15.122:0/1978961745 conn(0x7f62d80b62a0 msgr2=0x7f62d80b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:54.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dfac7640 1 -- 172.21.15.122:0/1978961745 shutdown_connections 2024-03-16T20:45:54.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dfac7640 1 -- 172.21.15.122:0/1978961745 wait complete. 2024-03-16T20:45:54.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dfac7640 1 Processor -- start 2024-03-16T20:45:54.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dfac7640 1 -- start start 2024-03-16T20:45:54.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dfac7640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62d8071a10 0x7f62d81e0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dfac7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8071df0 0x7f62d8143340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dfac7640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62d8148bd0 0x7f62d8132a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dfac7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d813ab90 con 0x7f62d8071df0 2024-03-16T20:45:54.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dfac7640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d813afb0 con 0x7f62d8148bd0 2024-03-16T20:45:54.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.243+0000 7f62dfac7640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d8135a20 con 0x7f62d8071a10 2024-03-16T20:45:54.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62dd062640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8071df0 0x7f62d8143340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62dd062640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8071df0 0x7f62d8143340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49252/0 (socket says 172.21.15.122:49252) 2024-03-16T20:45:54.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62dd062640 1 -- 172.21.15.122:0/1306293586 learned_addr learned my addr 172.21.15.122:0/1306293586 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:54.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62de064640 1 --2- 172.21.15.122:0/1306293586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62d8148bd0 0x7f62d8132a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62dd863640 1 --2- 172.21.15.122:0/1306293586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62d8071a10 0x7f62d81e0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62dd062640 1 -- 172.21.15.122:0/1306293586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62d8071a10 msgr2=0x7f62d81e0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:54.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62dd062640 1 --2- 172.21.15.122:0/1306293586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62d8071a10 0x7f62d81e0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62dd062640 1 -- 172.21.15.122:0/1306293586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62d8148bd0 msgr2=0x7f62d8132a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:54.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62dd062640 1 --2- 172.21.15.122:0/1306293586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62d8148bd0 0x7f62d8132a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62dd062640 1 -- 172.21.15.122:0/1306293586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c80094d0 con 0x7f62d8071df0 2024-03-16T20:45:54.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62dd062640 1 --2- 172.21.15.122:0/1306293586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8071df0 0x7f62d8143340 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f62c8019130 tx=0x7f62c801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:54.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62ce7fc640 1 -- 172.21.15.122:0/1306293586 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c80a9400 con 0x7f62d8071df0 2024-03-16T20:45:54.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62ce7fc640 1 -- 172.21.15.122:0/1306293586 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c80b1800 con 0x7f62d8071df0 2024-03-16T20:45:54.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62ce7fc640 1 -- 172.21.15.122:0/1306293586 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c80b1b70 con 0x7f62d8071df0 2024-03-16T20:45:54.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62dfac7640 1 -- 172.21.15.122:0/1306293586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d8142dc0 con 0x7f62d8071df0 2024-03-16T20:45:54.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62dfac7640 1 -- 172.21.15.122:0/1306293586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d81e1170 con 0x7f62d8071df0 2024-03-16T20:45:54.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62dfac7640 1 -- 172.21.15.122:0/1306293586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d813acd0 con 0x7f62d8071df0 2024-03-16T20:45:54.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62ce7fc640 1 -- 172.21.15.122:0/1306293586 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f62c80b1cd0 con 0x7f62d8071df0 2024-03-16T20:45:54.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62ce7fc640 1 --2- 172.21.15.122:0/1306293586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f62bc066e40 0x7f62bc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:54.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62ce7fc640 1 -- 172.21.15.122:0/1306293586 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62c811f5d0 con 0x7f62d8071df0 2024-03-16T20:45:54.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.247+0000 7f62dd863640 1 --2- 172.21.15.122:0/1306293586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f62bc066e40 0x7f62bc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:54.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.251+0000 7f62dd863640 1 --2- 172.21.15.122:0/1306293586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f62bc066e40 0x7f62bc069300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f62d4000f70 tx=0x7f62d4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:54.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.251+0000 7f62ce7fc640 1 -- 172.21.15.122:0/1306293586 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f62d813acd0 con 0x7f62d8071df0 2024-03-16T20:45:54.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.431+0000 7f62dfac7640 1 -- 172.21.15.122:0/1306293586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f62d813acd0 con 0x7f62d8071df0 2024-03-16T20:45:54.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.431+0000 7f62ce7fc640 1 -- 172.21.15.122:0/1306293586 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7f62d813acd0 con 0x7f62d8071df0 2024-03-16T20:45:54.436 INFO:teuthology.orchestra.run.smithi122.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-16T20:45:54.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.435+0000 7f62dfac7640 1 -- 172.21.15.122:0/1306293586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f62bc066e40 msgr2=0x7f62bc069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:54.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.435+0000 7f62dfac7640 1 --2- 172.21.15.122:0/1306293586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f62bc066e40 0x7f62bc069300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f62d4000f70 tx=0x7f62d4009040 comp rx=0 tx=0).stop 2024-03-16T20:45:54.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.435+0000 7f62dfac7640 1 -- 172.21.15.122:0/1306293586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8071df0 msgr2=0x7f62d8143340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:54.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.435+0000 7f62dfac7640 1 --2- 172.21.15.122:0/1306293586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8071df0 0x7f62d8143340 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f62c8019130 tx=0x7f62c801ad60 comp rx=0 tx=0).stop 2024-03-16T20:45:54.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.435+0000 7f62dfac7640 1 -- 172.21.15.122:0/1306293586 shutdown_connections 2024-03-16T20:45:54.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.435+0000 7f62dfac7640 1 --2- 172.21.15.122:0/1306293586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f62bc066e40 0x7f62bc069300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.435+0000 7f62dfac7640 1 --2- 172.21.15.122:0/1306293586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f62d8148bd0 0x7f62d8132a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.435+0000 7f62dfac7640 1 --2- 172.21.15.122:0/1306293586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f62d8071df0 0x7f62d8143340 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.435+0000 7f62dfac7640 1 --2- 172.21.15.122:0/1306293586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f62d8071a10 0x7f62d81e0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:54.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.435+0000 7f62dfac7640 1 -- 172.21.15.122:0/1306293586 >> 172.21.15.122:0/1306293586 conn(0x7f62d80b62a0 msgr2=0x7f62d806c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:54.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.435+0000 7f62dfac7640 1 -- 172.21.15.122:0/1306293586 shutdown_connections 2024-03-16T20:45:54.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:54.435+0000 7f62dfac7640 1 -- 172.21.15.122:0/1306293586 wait complete. 2024-03-16T20:45:54.453 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-16T20:45:54.453 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-16T20:45:54.453 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi122.front.sepia.ceph.com 2024-03-16T20:45:54.453 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-16T20:45:54.453 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-16T20:45:54.453 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-16T20:45:54.453 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-16T20:45:54.453 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-16T20:45:54.453 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-16T20:45:54.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:54.454 DEBUG:teuthology.orchestra.run.smithi122:> ip addr 2024-03-16T20:45:54.459 INFO:teuthology.orchestra.run.smithi122.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-16T20:45:54.459 INFO:teuthology.orchestra.run.smithi122.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-16T20:45:54.459 INFO:teuthology.orchestra.run.smithi122.stdout: inet 127.0.0.1/8 scope host lo 2024-03-16T20:45:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-03-16T20:45:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 ::1/128 scope host 2024-03-16T20:45:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-03-16T20:45:54.460 INFO:teuthology.orchestra.run.smithi122.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-16T20:45:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e4 brd ff:ff:ff:ff:ff:ff 2024-03-16T20:45:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f0 2024-03-16T20:45:54.460 INFO:teuthology.orchestra.run.smithi122.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-16T20:45:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:d8:d9:e5 brd ff:ff:ff:ff:ff:ff 2024-03-16T20:45:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: altname enp7s0f1 2024-03-16T20:45:54.460 INFO:teuthology.orchestra.run.smithi122.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-03-16T20:45:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:14 brd ff:ff:ff:ff:ff:ff 2024-03-16T20:45:54.460 INFO:teuthology.orchestra.run.smithi122.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-16T20:45:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0c:c4:7a:8f:d5:15 brd ff:ff:ff:ff:ff:ff 2024-03-16T20:45:54.460 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.21.15.122/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-03-16T20:45:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft 42489sec preferred_lft 42489sec 2024-03-16T20:45:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::ec4:7aff:fe8f:d515/64 scope link 2024-03-16T20:45:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-03-16T20:45:54.461 INFO:teuthology.orchestra.run.smithi122.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-03-16T20:45:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 02:42:0d:60:22:73 brd ff:ff:ff:ff:ff:ff 2024-03-16T20:45:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-03-16T20:45:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-03-16T20:45:54.461 INFO:teuthology.orchestra.run.smithi122.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-03-16T20:45:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-03-16T20:45:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-03-16T20:45:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-03-16T20:45:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-03-16T20:45:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-03-16T20:45:54.461 INFO:teuthology.orchestra.run.smithi122.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-03-16T20:45:54.461 INFO:teuthology.orchestra.run.smithi122.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T20:45:54.462 INFO:teuthology.orchestra.run.smithi122.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-03-16T20:45:54.462 INFO:teuthology.orchestra.run.smithi122.stdout: valid_lft forever preferred_lft forever 2024-03-16T20:45:54.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:54.462 DEBUG:teuthology.orchestra.run.smithi122:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-16T20:45:54.514 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-03-16T20:45:54.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:54.516 DEBUG:teuthology.orchestra.run.smithi122:> route 2024-03-16T20:45:54.568 INFO:teuthology.orchestra.run.smithi122.stdout:Kernel IP routing table 2024-03-16T20:45:54.568 INFO:teuthology.orchestra.run.smithi122.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-16T20:45:54.568 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-03-16T20:45:54.568 INFO:teuthology.orchestra.run.smithi122.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-16T20:45:54.568 INFO:teuthology.orchestra.run.smithi122.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-03-16T20:45:54.568 INFO:teuthology.orchestra.run.smithi122.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-03-16T20:45:54.568 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-16T20:45:54.568 INFO:teuthology.orchestra.run.smithi122.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-16T20:45:54.568 INFO:teuthology.orchestra.run.smithi122.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-16T20:45:54.568 INFO:teuthology.orchestra.run.smithi122.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-03-16T20:45:54.569 INFO:teuthology.orchestra.run.smithi122.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-16T20:45:54.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:54.569 DEBUG:teuthology.orchestra.run.smithi122:> (cd / && exec bash -c ' 2024-03-16T20:45:54.569 DEBUG:teuthology.orchestra.run.smithi122:> set -e 2024-03-16T20:45:54.569 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-16T20:45:54.569 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-16T20:45:54.569 DEBUG:teuthology.orchestra.run.smithi122:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-16T20:45:54.569 DEBUG:teuthology.orchestra.run.smithi122:> ') 2024-03-16T20:45:54.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:54.637 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-03-16T20:45:54.681 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-16T20:45:54.682 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-16T20:45:54.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:54.682 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link set brx.0 up 2024-03-16T20:45:54.738 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-03-16T20:45:54.738 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T20:45:54.738 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-16T20:45:54.787 INFO:teuthology.orchestra.run.smithi122.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-16T20:45:54.787 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-16T20:45:54.788 DEBUG:teuthology.orchestra.run.smithi122:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-16T20:45:54.835 DEBUG:teuthology.orchestra.run.smithi122:> sudo modprobe fuse 2024-03-16T20:45:54.891 DEBUG:teuthology.orchestra.run.smithi122:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T20:45:54.938 INFO:teuthology.orchestra.run.smithi122.stdout:/sys 2024-03-16T20:45:54.938 INFO:teuthology.orchestra.run.smithi122.stdout:/proc 2024-03-16T20:45:54.938 INFO:teuthology.orchestra.run.smithi122.stdout:/dev 2024-03-16T20:45:54.938 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/pts 2024-03-16T20:45:54.938 INFO:teuthology.orchestra.run.smithi122.stdout:/run 2024-03-16T20:45:54.938 INFO:teuthology.orchestra.run.smithi122.stdout:/ 2024-03-16T20:45:54.939 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/security 2024-03-16T20:45:54.939 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/shm 2024-03-16T20:45:54.939 INFO:teuthology.orchestra.run.smithi122.stdout:/run/lock 2024-03-16T20:45:54.939 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/cgroup 2024-03-16T20:45:54.939 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/pstore 2024-03-16T20:45:54.939 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/bpf 2024-03-16T20:45:54.939 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T20:45:54.939 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/mqueue 2024-03-16T20:45:54.939 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/hugepages 2024-03-16T20:45:54.939 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/debug 2024-03-16T20:45:54.939 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/tracing 2024-03-16T20:45:54.939 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/fs/nfsd 2024-03-16T20:45:54.940 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/fuse/connections 2024-03-16T20:45:54.940 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/config 2024-03-16T20:45:54.940 INFO:teuthology.orchestra.run.smithi122.stdout:/run/credentials/systemd-sysusers.service 2024-03-16T20:45:54.940 INFO:teuthology.orchestra.run.smithi122.stdout:/run/rpc_pipefs 2024-03-16T20:45:54.940 INFO:teuthology.orchestra.run.smithi122.stdout:/run/user/1000 2024-03-16T20:45:54.940 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph 2024-03-16T20:45:54.940 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-0 2024-03-16T20:45:54.940 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-1 2024-03-16T20:45:54.940 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-2 2024-03-16T20:45:54.940 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-3 2024-03-16T20:45:54.940 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns 2024-03-16T20:45:54.940 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T20:45:54.940 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T20:45:54.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:54.941 DEBUG:teuthology.orchestra.run.smithi122:> ls /sys/fs/fuse/connections 2024-03-16T20:45:54.986 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-03-16T20:45:54.986 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-03-16T20:45:55.031 DEBUG:teuthology.orchestra.run.smithi122:> sudo modprobe fuse 2024-03-16T20:45:55.043 DEBUG:teuthology.orchestra.run.smithi122:> cat /proc/self/mounts | awk '{print $2}' 2024-03-16T20:45:55.095 INFO:teuthology.orchestra.run.smithi122.stdout:/sys 2024-03-16T20:45:55.095 INFO:teuthology.orchestra.run.smithi122.stdout:/proc 2024-03-16T20:45:55.095 INFO:teuthology.orchestra.run.smithi122.stdout:/dev 2024-03-16T20:45:55.095 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/pts 2024-03-16T20:45:55.095 INFO:teuthology.orchestra.run.smithi122.stdout:/run 2024-03-16T20:45:55.095 INFO:teuthology.orchestra.run.smithi122.stdout:/ 2024-03-16T20:45:55.095 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/security 2024-03-16T20:45:55.095 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/shm 2024-03-16T20:45:55.095 INFO:teuthology.orchestra.run.smithi122.stdout:/run/lock 2024-03-16T20:45:55.096 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/cgroup 2024-03-16T20:45:55.096 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/pstore 2024-03-16T20:45:55.096 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/bpf 2024-03-16T20:45:55.096 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/sys/fs/binfmt_misc 2024-03-16T20:45:55.096 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/mqueue 2024-03-16T20:45:55.096 INFO:teuthology.orchestra.run.smithi122.stdout:/dev/hugepages 2024-03-16T20:45:55.096 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/debug 2024-03-16T20:45:55.096 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/tracing 2024-03-16T20:45:55.096 INFO:teuthology.orchestra.run.smithi122.stdout:/proc/fs/nfsd 2024-03-16T20:45:55.096 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/fs/fuse/connections 2024-03-16T20:45:55.096 INFO:teuthology.orchestra.run.smithi122.stdout:/sys/kernel/config 2024-03-16T20:45:55.096 INFO:teuthology.orchestra.run.smithi122.stdout:/run/credentials/systemd-sysusers.service 2024-03-16T20:45:55.096 INFO:teuthology.orchestra.run.smithi122.stdout:/run/rpc_pipefs 2024-03-16T20:45:55.096 INFO:teuthology.orchestra.run.smithi122.stdout:/run/user/1000 2024-03-16T20:45:55.097 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph 2024-03-16T20:45:55.097 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-0 2024-03-16T20:45:55.097 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-1 2024-03-16T20:45:55.097 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-2 2024-03-16T20:45:55.097 INFO:teuthology.orchestra.run.smithi122.stdout:/var/lib/ceph/osd/ceph-3 2024-03-16T20:45:55.097 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns 2024-03-16T20:45:55.097 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T20:45:55.097 INFO:teuthology.orchestra.run.smithi122.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T20:45:55.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:55.097 DEBUG:teuthology.orchestra.run.smithi122:> ls /sys/fs/fuse/connections 2024-03-16T20:45:55.098 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi122.stderr:2024-03-16T20:45:55.091+0000 7f879e700580 -1 init, newargv = 0x56450e5ff990 newargc=15 2024-03-16T20:45:55.098 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi122.stderr:ceph-fuse[36767]: starting ceph client 2024-03-16T20:45:55.100 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi122.stderr:ceph-fuse[36767]: starting fuse 2024-03-16T20:45:55.103 INFO:teuthology.orchestra.run.smithi122.stdout:45 2024-03-16T20:45:55.103 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-03-16T20:45:55.103 DEBUG:teuthology.orchestra.run.smithi122:> sudo stdin-killer -- python3 -c ' 2024-03-16T20:45:55.103 DEBUG:teuthology.orchestra.run.smithi122:> import glob 2024-03-16T20:45:55.103 DEBUG:teuthology.orchestra.run.smithi122:> import re 2024-03-16T20:45:55.103 DEBUG:teuthology.orchestra.run.smithi122:> import os 2024-03-16T20:45:55.103 DEBUG:teuthology.orchestra.run.smithi122:> import subprocess 2024-03-16T20:45:55.103 DEBUG:teuthology.orchestra.run.smithi122:> 2024-03-16T20:45:55.103 DEBUG:teuthology.orchestra.run.smithi122:> def _find_admin_socket(client_name): 2024-03-16T20:45:55.104 DEBUG:teuthology.orchestra.run.smithi122:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-03-16T20:45:55.104 DEBUG:teuthology.orchestra.run.smithi122:> files = glob.glob(asok_path) 2024-03-16T20:45:55.104 DEBUG:teuthology.orchestra.run.smithi122:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-03-16T20:45:55.104 DEBUG:teuthology.orchestra.run.smithi122:> 2024-03-16T20:45:55.104 DEBUG:teuthology.orchestra.run.smithi122:> # Given a non-glob path, it better be there 2024-03-16T20:45:55.104 DEBUG:teuthology.orchestra.run.smithi122:> if "*" not in asok_path: 2024-03-16T20:45:55.104 DEBUG:teuthology.orchestra.run.smithi122:> assert(len(files) == 1) 2024-03-16T20:45:55.104 DEBUG:teuthology.orchestra.run.smithi122:> return files[0] 2024-03-16T20:45:55.104 DEBUG:teuthology.orchestra.run.smithi122:> 2024-03-16T20:45:55.104 DEBUG:teuthology.orchestra.run.smithi122:> for f in files: 2024-03-16T20:45:55.104 DEBUG:teuthology.orchestra.run.smithi122:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-03-16T20:45:55.104 DEBUG:teuthology.orchestra.run.smithi122:> if os.path.exists("/proc/{0}".format(pid)): 2024-03-16T20:45:55.104 DEBUG:teuthology.orchestra.run.smithi122:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-03-16T20:45:55.104 DEBUG:teuthology.orchestra.run.smithi122:> contents = proc_f.read() 2024-03-16T20:45:55.104 DEBUG:teuthology.orchestra.run.smithi122:> if mountpoint in contents: 2024-03-16T20:45:55.105 DEBUG:teuthology.orchestra.run.smithi122:> return f 2024-03-16T20:45:55.105 DEBUG:teuthology.orchestra.run.smithi122:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-03-16T20:45:55.105 DEBUG:teuthology.orchestra.run.smithi122:> 2024-03-16T20:45:55.105 DEBUG:teuthology.orchestra.run.smithi122:> print(_find_admin_socket("client.0")) 2024-03-16T20:45:55.105 DEBUG:teuthology.orchestra.run.smithi122:> ' 2024-03-16T20:45:55.208 INFO:teuthology.orchestra.run.smithi122.stdout:/var/run/ceph/ceph-client.0.36767.asok 2024-03-16T20:45:55.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-03-16T20:45:55.219 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36767.asok 2024-03-16T20:45:55.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:55.219 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36767.asok status 2024-03-16T20:45:55.363 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-16T20:45:55.363 INFO:teuthology.orchestra.run.smithi122.stdout: "metadata": { 2024-03-16T20:45:55.363 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_sha1": "aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b", 2024-03-16T20:45:55.363 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_version": "ceph version 19.0.0-1681-gaba3c1d2 (aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b) squid (dev)", 2024-03-16T20:45:55.363 INFO:teuthology.orchestra.run.smithi122.stdout: "entity_id": "0", 2024-03-16T20:45:55.363 INFO:teuthology.orchestra.run.smithi122.stdout: "hostname": "smithi122", 2024-03-16T20:45:55.364 INFO:teuthology.orchestra.run.smithi122.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-03-16T20:45:55.364 INFO:teuthology.orchestra.run.smithi122.stdout: "pid": "36767", 2024-03-16T20:45:55.364 INFO:teuthology.orchestra.run.smithi122.stdout: "root": "/" 2024-03-16T20:45:55.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:55.364 INFO:teuthology.orchestra.run.smithi122.stdout: "dentry_count": 0, 2024-03-16T20:45:55.364 INFO:teuthology.orchestra.run.smithi122.stdout: "dentry_pinned_count": 0, 2024-03-16T20:45:55.364 INFO:teuthology.orchestra.run.smithi122.stdout: "id": 5223, 2024-03-16T20:45:55.364 INFO:teuthology.orchestra.run.smithi122.stdout: "inst": { 2024-03-16T20:45:55.364 INFO:teuthology.orchestra.run.smithi122.stdout: "name": { 2024-03-16T20:45:55.364 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "client", 2024-03-16T20:45:55.364 INFO:teuthology.orchestra.run.smithi122.stdout: "num": 5223 2024-03-16T20:45:55.364 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:55.364 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": { 2024-03-16T20:45:55.364 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v1", 2024-03-16T20:45:55.364 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "192.168.0.1:0", 2024-03-16T20:45:55.365 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 3830453124 2024-03-16T20:45:55.365 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-03-16T20:45:55.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:55.365 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": { 2024-03-16T20:45:55.365 INFO:teuthology.orchestra.run.smithi122.stdout: "type": "v1", 2024-03-16T20:45:55.365 INFO:teuthology.orchestra.run.smithi122.stdout: "addr": "192.168.0.1:0", 2024-03-16T20:45:55.365 INFO:teuthology.orchestra.run.smithi122.stdout: "nonce": 3830453124 2024-03-16T20:45:55.365 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-03-16T20:45:55.365 INFO:teuthology.orchestra.run.smithi122.stdout: "inst_str": "client.5223 192.168.0.1:0/3830453124", 2024-03-16T20:45:55.365 INFO:teuthology.orchestra.run.smithi122.stdout: "addr_str": "192.168.0.1:0/3830453124", 2024-03-16T20:45:55.366 INFO:teuthology.orchestra.run.smithi122.stdout: "inode_count": 1, 2024-03-16T20:45:55.366 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_epoch": 13, 2024-03-16T20:45:55.366 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_epoch": 30, 2024-03-16T20:45:55.366 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_epoch_barrier": 0, 2024-03-16T20:45:55.366 INFO:teuthology.orchestra.run.smithi122.stdout: "blocklisted": false, 2024-03-16T20:45:55.366 INFO:teuthology.orchestra.run.smithi122.stdout: "fs_name": "cephfs" 2024-03-16T20:45:55.366 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-16T20:45:55.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:55.376 DEBUG:teuthology.orchestra.run.smithi122:> stat --file-system '--printf=%T 2024-03-16T20:45:55.376 DEBUG:teuthology.orchestra.run.smithi122:> ' -- /home/ubuntu/cephtest/mnt.0 2024-03-16T20:45:55.382 INFO:teuthology.orchestra.run.smithi122.stdout:fuseblk 2024-03-16T20:45:55.382 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-03-16T20:45:55.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:55.383 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-16T20:45:55.443 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-03-16T20:45:55.491 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.eOee0YNqa0 2024-03-16T20:45:55.491 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:45:55.491 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/tmp/tmp.eOee0YNqa0 2024-03-16T20:45:55.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:55.538 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'mkdir -p dir1/dir2/dir3') 2024-03-16T20:45:55.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:55.655 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'chmod 755 /tmp/test-cephfs9i_49ozq') 2024-03-16T20:45:55.663 INFO:tasks.cephfs.test_cephfs_shell:Running script "/tmp/test-cephfs9i_49ozq" 2024-03-16T20:45:55.663 DEBUG:teuthology.orchestra.run.smithi122:> cephfs-shell -b /tmp/test-cephfs9i_49ozq 2024-03-16T20:45:56.025 INFO:teuthology.orchestra.run.smithi122.stderr:Unrecognized command 2024-03-16T20:45:56.054 DEBUG:teuthology.orchestra.run:got remote process result: 127 2024-03-16T20:45:56.054 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-03-16T20:45:56.055 DEBUG:teuthology.orchestra.run.smithi122:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-16T20:45:56.100 DEBUG:teuthology.orchestra.run.smithi195:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-03-16T20:45:56.146 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-03-16T20:45:56.146 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:45:56.146 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T20:45:56.152 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:56.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.299+0000 7fbca239a640 1 Processor -- start 2024-03-16T20:45:56.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.299+0000 7fbca239a640 1 -- start start 2024-03-16T20:45:56.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.303+0000 7fbca239a640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fbc9c072d60 0x7fbc9c1432c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:56.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.303+0000 7fbca239a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fbc9c143800 0x7fbc9c14e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:56.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.303+0000 7fbca239a640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc9c09a2b0 con 0x7fbc9c066fa0 2024-03-16T20:45:56.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.303+0000 7fbca239a640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc9c0b4db0 con 0x7fbc9c072d60 2024-03-16T20:45:56.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.303+0000 7fbca239a640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc9c09bc30 con 0x7fbc9c143800 2024-03-16T20:45:56.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.303+0000 7fbc9b7fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fbc9c072d60 0x7fbc9c1432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:56.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.303+0000 7fbc9b7fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fbc9c072d60 0x7fbc9c1432c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49274/0 (socket says 172.21.15.122:49274) 2024-03-16T20:45:56.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.303+0000 7fbc9b7fe640 1 -- 172.21.15.122:0/4096478848 learned_addr learned my addr 172.21.15.122:0/4096478848 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:56.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.303+0000 7fbc9b7fe640 1 -- 172.21.15.122:0/4096478848 >> v2:172.21.15.195:3300/0 conn(0x7fbc9c143800 msgr2=0x7fbc9c14e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:56.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.303+0000 7fbc9b7fe640 1 --2- 172.21.15.122:0/4096478848 >> v2:172.21.15.195:3300/0 conn(0x7fbc9c143800 0x7fbc9c14e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.303+0000 7fbc9b7fe640 1 -- 172.21.15.122:0/4096478848 >> v1:172.21.15.195:6789/0 conn(0x7fbc9c066fa0 legacy=0x7fbc9c0af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:56.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.303+0000 7fbc9b7fe640 1 -- 172.21.15.122:0/4096478848 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c09a040 con 0x7fbc9c072d60 2024-03-16T20:45:56.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.303+0000 7fbc9b7fe640 1 --2- 172.21.15.122:0/4096478848 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc9c072d60 0x7fbc9c1432c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c148490 tx=0x7fbc9001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb4357137740ca6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:56.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.303+0000 7fbc9affd640 1 -- 172.21.15.122:0/4096478848 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc900ab600 con 0x7fbc9c072d60 2024-03-16T20:45:56.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.303+0000 7fbc9affd640 1 -- 172.21.15.122:0/4096478848 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc900a7050 con 0x7fbc9c072d60 2024-03-16T20:45:56.310 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.303+0000 7fbc9affd640 1 -- 172.21.15.122:0/4096478848 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc900ab980 con 0x7fbc9c072d60 2024-03-16T20:45:56.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 -- 172.21.15.122:0/4096478848 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc9c072d60 msgr2=0x7fbc9c1432c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:56.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 --2- 172.21.15.122:0/4096478848 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc9c072d60 0x7fbc9c1432c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c148490 tx=0x7fbc9001c450 comp rx=0 tx=0).stop 2024-03-16T20:45:56.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 -- 172.21.15.122:0/4096478848 shutdown_connections 2024-03-16T20:45:56.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 --2- 172.21.15.122:0/4096478848 >> v2:172.21.15.195:3300/0 conn(0x7fbc9c143800 0x7fbc9c14e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 --2- 172.21.15.122:0/4096478848 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc9c072d60 0x7fbc9c1432c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 -- 172.21.15.122:0/4096478848 >> 172.21.15.122:0/4096478848 conn(0x7fbc9c06d320 msgr2=0x7fbc9c06d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:56.311 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 -- 172.21.15.122:0/4096478848 shutdown_connections 2024-03-16T20:45:56.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 -- 172.21.15.122:0/4096478848 wait complete. 2024-03-16T20:45:56.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 Processor -- start 2024-03-16T20:45:56.312 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 -- start start 2024-03-16T20:45:56.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc9c066fa0 0x7fbc9c1e3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:56.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc9c072d60 0x7fbc9c1df720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:56.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc9c143800 0x7fbc9c146cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:56.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c148070 con 0x7fbc9c072d60 2024-03-16T20:45:56.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c1e2c90 con 0x7fbc9c066fa0 2024-03-16T20:45:56.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc9c1e2f70 con 0x7fbc9c143800 2024-03-16T20:45:56.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbc9b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc9c072d60 0x7fbc9c1df720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:56.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbc9b7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc9c072d60 0x7fbc9c1df720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49280/0 (socket says 172.21.15.122:49280) 2024-03-16T20:45:56.315 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbc9b7fe640 1 -- 172.21.15.122:0/1066268586 learned_addr learned my addr 172.21.15.122:0/1066268586 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:56.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbc9bfff640 1 --2- 172.21.15.122:0/1066268586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc9c066fa0 0x7fbc9c1e3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:56.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca0937640 1 --2- 172.21.15.122:0/1066268586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc9c143800 0x7fbc9c146cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:56.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbc9b7fe640 1 -- 172.21.15.122:0/1066268586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc9c143800 msgr2=0x7fbc9c146cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:56.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbc9b7fe640 1 --2- 172.21.15.122:0/1066268586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc9c143800 0x7fbc9c146cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbc9b7fe640 1 -- 172.21.15.122:0/1066268586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc9c066fa0 msgr2=0x7fbc9c1e3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:56.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbc9b7fe640 1 --2- 172.21.15.122:0/1066268586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc9c066fa0 0x7fbc9c1e3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbc9b7fe640 1 -- 172.21.15.122:0/1066268586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc900094d0 con 0x7fbc9c072d60 2024-03-16T20:45:56.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbc9bfff640 1 --2- 172.21.15.122:0/1066268586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc9c066fa0 0x7fbc9c1e3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:56.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca0937640 1 --2- 172.21.15.122:0/1066268586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc9c143800 0x7fbc9c146cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:56.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbc9b7fe640 1 --2- 172.21.15.122:0/1066268586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc9c072d60 0x7fbc9c1df720 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbc900ae040 tx=0x7fbc9001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:56.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbc98ff9640 1 -- 172.21.15.122:0/1066268586 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc900b6830 con 0x7fbc9c072d60 2024-03-16T20:45:56.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbc98ff9640 1 -- 172.21.15.122:0/1066268586 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc900b69b0 con 0x7fbc9c072d60 2024-03-16T20:45:56.316 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 -- 172.21.15.122:0/1066268586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc9c1e3250 con 0x7fbc9c072d60 2024-03-16T20:45:56.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbc98ff9640 1 -- 172.21.15.122:0/1066268586 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc900b6d60 con 0x7fbc9c072d60 2024-03-16T20:45:56.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 -- 172.21.15.122:0/1066268586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc9c148270 con 0x7fbc9c072d60 2024-03-16T20:45:56.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.307+0000 7fbca239a640 1 -- 172.21.15.122:0/1066268586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc9c0b4020 con 0x7fbc9c072d60 2024-03-16T20:45:56.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.311+0000 7fbc98ff9640 1 -- 172.21.15.122:0/1066268586 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fbc900b2040 con 0x7fbc9c072d60 2024-03-16T20:45:56.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.311+0000 7fbc98ff9640 1 --2- 172.21.15.122:0/1066268586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fbc88066e70 0x7fbc88069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:56.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.311+0000 7fbc98ff9640 1 -- 172.21.15.122:0/1066268586 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc9011f0c0 con 0x7fbc9c072d60 2024-03-16T20:45:56.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.311+0000 7fbc9bfff640 1 --2- 172.21.15.122:0/1066268586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fbc88066e70 0x7fbc88069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:56.317 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.311+0000 7fbc9bfff640 1 --2- 172.21.15.122:0/1066268586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fbc88066e70 0x7fbc88069330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0016e0 tx=0x7fbc8c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:56.319 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.315+0000 7fbc98ff9640 1 -- 172.21.15.122:0/1066268586 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fbc900ebf40 con 0x7fbc9c072d60 2024-03-16T20:45:56.652 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.647+0000 7fbca239a640 1 -- 172.21.15.122:0/1066268586 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc9c075280 con 0x7fbc9c072d60 2024-03-16T20:45:56.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.651+0000 7fbc98ff9640 1 -- 172.21.15.122:0/1066268586 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+16705 (secure 0 0 0) 0x7fbc900ef600 con 0x7fbc9c072d60 2024-03-16T20:45:56.655 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:56.655 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:45.942613+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:42.678541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:43.274469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:56.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.651+0000 7fbca239a640 1 -- 172.21.15.122:0/1066268586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fbc88066e70 msgr2=0x7fbc88069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:56.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.651+0000 7fbca239a640 1 --2- 172.21.15.122:0/1066268586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fbc88066e70 0x7fbc88069330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c0016e0 tx=0x7fbc8c009040 comp rx=0 tx=0).stop 2024-03-16T20:45:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.651+0000 7fbca239a640 1 -- 172.21.15.122:0/1066268586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc9c072d60 msgr2=0x7fbc9c1df720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.651+0000 7fbca239a640 1 --2- 172.21.15.122:0/1066268586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc9c072d60 0x7fbc9c1df720 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbc900ae040 tx=0x7fbc9001ac90 comp rx=0 tx=0).stop 2024-03-16T20:45:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.651+0000 7fbca239a640 1 -- 172.21.15.122:0/1066268586 shutdown_connections 2024-03-16T20:45:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.651+0000 7fbca239a640 1 --2- 172.21.15.122:0/1066268586 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fbc88066e70 0x7fbc88069330 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.651+0000 7fbca239a640 1 --2- 172.21.15.122:0/1066268586 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fbc9c143800 0x7fbc9c146cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.651+0000 7fbca239a640 1 --2- 172.21.15.122:0/1066268586 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbc9c072d60 0x7fbc9c1df720 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.651+0000 7fbca239a640 1 --2- 172.21.15.122:0/1066268586 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fbc9c066fa0 0x7fbc9c1e3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.651+0000 7fbca239a640 1 -- 172.21.15.122:0/1066268586 >> 172.21.15.122:0/1066268586 conn(0x7fbc9c06d320 msgr2=0x7fbc9c0b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.655+0000 7fbca239a640 1 -- 172.21.15.122:0/1066268586 shutdown_connections 2024-03-16T20:45:56.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.655+0000 7fbca239a640 1 -- 172.21.15.122:0/1066268586 wait complete. 2024-03-16T20:45:56.681 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T20:44:52.653523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T20:45:42.678541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T20:45:43.274469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T20:45:56.682 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T20:45:56.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.787+0000 7fd8575dd640 1 Processor -- start 2024-03-16T20:45:56.794 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.787+0000 7fd8575dd640 1 -- start start 2024-03-16T20:45:56.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd8575dd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8500b7d40 0x7fd8500b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:56.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd8575dd640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8500b6740 con 0x7fd850148480 2024-03-16T20:45:56.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd8575dd640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8500b7920 con 0x7fd8500b7540 2024-03-16T20:45:56.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd8575dd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8500b23d0 con 0x7fd8500b7d40 2024-03-16T20:45:56.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd855b7a640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fd850148480 0x7fd85014f7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:37480/0 (socket says 172.21.15.122:37480) 2024-03-16T20:45:56.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd855b7a640 1 -- 172.21.15.122:0/2593252521 learned_addr learned my addr 172.21.15.122:0/2593252521 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:56.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd854b78640 1 --2- 172.21.15.122:0/2593252521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8500b7d40 0x7fd8500b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:56.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 -- 172.21.15.122:0/2593252521 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3987893049 0 0) 0x7fd8500b6740 con 0x7fd850148480 2024-03-16T20:45:56.796 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 -- 172.21.15.122:0/2593252521 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd83c003530 con 0x7fd850148480 2024-03-16T20:45:56.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 -- 172.21.15.122:0/2593252521 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3850058277 0 0) 0x7fd83c003530 con 0x7fd850148480 2024-03-16T20:45:56.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 -- 172.21.15.122:0/2593252521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8500b7d40 msgr2=0x7fd8500b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:56.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 --2- 172.21.15.122:0/2593252521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8500b7d40 0x7fd8500b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 -- 172.21.15.122:0/2593252521 >> v1:172.21.15.195:6789/0 conn(0x7fd8500b7540 legacy=0x7fd85013ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:56.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 -- 172.21.15.122:0/2593252521 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8500b31b0 con 0x7fd850148480 2024-03-16T20:45:56.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 -- 172.21.15.122:0/2593252521 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7fd848001a00 con 0x7fd850148480 2024-03-16T20:45:56.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 -- 172.21.15.122:0/2593252521 >> v1:172.21.15.122:6789/0 conn(0x7fd850148480 legacy=0x7fd85014f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:56.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 --2- 172.21.15.122:0/2593252521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8500b7d40 0x7fd83c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:56.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 --2- 172.21.15.122:0/2593252521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd83c003f30 0x7fd83c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:56.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 --2- 172.21.15.122:0/2593252521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd83c004d10 0x7fd83c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:56.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 -- 172.21.15.122:0/2593252521 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8500b23d0 con 0x7fd83c004d10 2024-03-16T20:45:56.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 -- 172.21.15.122:0/2593252521 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c00bb20 con 0x7fd83c003f30 2024-03-16T20:45:56.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 -- 172.21.15.122:0/2593252521 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c00bce0 con 0x7fd8500b7d40 2024-03-16T20:45:56.797 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd855b7a640 1 --2- 172.21.15.122:0/2593252521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd83c004d10 0x7fd83c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:56.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd855379640 1 --2- 172.21.15.122:0/2593252521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8500b7d40 0x7fd83c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:56.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd854b78640 1 --2- 172.21.15.122:0/2593252521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd83c003f30 0x7fd83c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:56.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd855b7a640 1 -- 172.21.15.122:0/2593252521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8500b7d40 msgr2=0x7fd83c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:56.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd855b7a640 1 --2- 172.21.15.122:0/2593252521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8500b7d40 0x7fd83c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd855b7a640 1 -- 172.21.15.122:0/2593252521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd83c003f30 msgr2=0x7fd83c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:56.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd855b7a640 1 --2- 172.21.15.122:0/2593252521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd83c003f30 0x7fd83c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd855b7a640 1 -- 172.21.15.122:0/2593252521 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c003530 con 0x7fd83c004d10 2024-03-16T20:45:56.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd855379640 1 --2- 172.21.15.122:0/2593252521 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd8500b7d40 0x7fd83c0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:56.798 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd854b78640 1 --2- 172.21.15.122:0/2593252521 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd83c003f30 0x7fd83c0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd855b7a640 1 --2- 172.21.15.122:0/2593252521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd83c004d10 0x7fd83c005190 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd84800a820 tx=0x7fd84801faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d629a35336bbcd3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 -- 172.21.15.122:0/2593252521 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480aa6d0 con 0x7fd83c004d10 2024-03-16T20:45:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 -- 172.21.15.122:0/2593252521 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8480aa830 con 0x7fd83c004d10 2024-03-16T20:45:56.801 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.791+0000 7fd847fff640 1 -- 172.21.15.122:0/2593252521 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480aac20 con 0x7fd83c004d10 2024-03-16T20:45:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd8575dd640 1 -- 172.21.15.122:0/2593252521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd83c004d10 msgr2=0x7fd83c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd8575dd640 1 --2- 172.21.15.122:0/2593252521 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd83c004d10 0x7fd83c005190 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd84800a820 tx=0x7fd84801faa0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd855b7a640 1 -- 172.21.15.122:0/2593252521 reap_dead start 2024-03-16T20:45:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd8575dd640 1 -- 172.21.15.122:0/2593252521 shutdown_connections 2024-03-16T20:45:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd8575dd640 1 -- 172.21.15.122:0/2593252521 >> 172.21.15.122:0/2593252521 conn(0x7fd85006fc00 msgr2=0x7fd850070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd8575dd640 1 -- 172.21.15.122:0/2593252521 shutdown_connections 2024-03-16T20:45:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd8575dd640 1 -- 172.21.15.122:0/2593252521 wait complete. 2024-03-16T20:45:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd8575dd640 1 Processor -- start 2024-03-16T20:45:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd8575dd640 1 -- start start 2024-03-16T20:45:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd8575dd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd85006d350 0x7fd8501dc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd8575dd640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8501dc9a0 0x7fd8501df9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd8575dd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8501dff10 0x7fd8501e0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:56.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd8575dd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd850145b80 con 0x7fd8501dff10 2024-03-16T20:45:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd8575dd640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85006d730 con 0x7fd8501dc9a0 2024-03-16T20:45:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd8575dd640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd850139610 con 0x7fd85006d350 2024-03-16T20:45:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd855b7a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8501dff10 0x7fd8501e0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd855b7a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8501dff10 0x7fd8501e0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49296/0 (socket says 172.21.15.122:49296) 2024-03-16T20:45:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd855b7a640 1 -- 172.21.15.122:0/2329617448 learned_addr learned my addr 172.21.15.122:0/2329617448 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd855379640 1 --2- 172.21.15.122:0/2329617448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd85006d350 0x7fd8501dc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd854b78640 1 --2- 172.21.15.122:0/2329617448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8501dc9a0 0x7fd8501df9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd855b7a640 1 -- 172.21.15.122:0/2329617448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd85006d350 msgr2=0x7fd8501dc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd855b7a640 1 --2- 172.21.15.122:0/2329617448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd85006d350 0x7fd8501dc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd855b7a640 1 -- 172.21.15.122:0/2329617448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8501dc9a0 msgr2=0x7fd8501df9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd855b7a640 1 --2- 172.21.15.122:0/2329617448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8501dc9a0 0x7fd8501df9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd855b7a640 1 -- 172.21.15.122:0/2329617448 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85006dc50 con 0x7fd8501dff10 2024-03-16T20:45:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd855b7a640 1 --2- 172.21.15.122:0/2329617448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8501dff10 0x7fd8501e0390 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd84801f8d0 tx=0x7fd83c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd845ffb640 1 -- 172.21.15.122:0/2329617448 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8480aa6d0 con 0x7fd8501dff10 2024-03-16T20:45:56.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd845ffb640 1 -- 172.21.15.122:0/2329617448 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd848001fa0 con 0x7fd8501dff10 2024-03-16T20:45:56.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd845ffb640 1 -- 172.21.15.122:0/2329617448 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd848001a00 con 0x7fd8501dff10 2024-03-16T20:45:56.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd8575dd640 1 -- 172.21.15.122:0/2329617448 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd850138730 con 0x7fd8501dff10 2024-03-16T20:45:56.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.795+0000 7fd8575dd640 1 -- 172.21.15.122:0/2329617448 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8501dce20 con 0x7fd8501dff10 2024-03-16T20:45:56.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.799+0000 7fd8575dd640 1 -- 172.21.15.122:0/2329617448 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd850145d30 con 0x7fd8501dff10 2024-03-16T20:45:56.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.799+0000 7fd845ffb640 1 -- 172.21.15.122:0/2329617448 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fd8480c4450 con 0x7fd8501dff10 2024-03-16T20:45:56.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.799+0000 7fd845ffb640 1 --2- 172.21.15.122:0/2329617448 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd834066e40 0x7fd834069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:56.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.799+0000 7fd845ffb640 1 -- 172.21.15.122:0/2329617448 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd848122040 con 0x7fd8501dff10 2024-03-16T20:45:56.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.799+0000 7fd855379640 1 --2- 172.21.15.122:0/2329617448 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd834066e40 0x7fd834069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:56.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.799+0000 7fd855379640 1 --2- 172.21.15.122:0/2329617448 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd834066e40 0x7fd834069300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd84c007fd0 tx=0x7fd84c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:56.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.803+0000 7fd845ffb640 1 -- 172.21.15.122:0/2329617448 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd850145d30 con 0x7fd8501dff10 2024-03-16T20:45:56.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.979+0000 7fd8575dd640 1 -- 172.21.15.122:0/2329617448 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd850145d30 con 0x7fd8501dff10 2024-03-16T20:45:56.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.979+0000 7fd845ffb640 1 -- 172.21.15.122:0/2329617448 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd850145d30 con 0x7fd8501dff10 2024-03-16T20:45:56.983 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-03-16T20:45:56.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.979+0000 7fd8575dd640 1 -- 172.21.15.122:0/2329617448 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd834066e40 msgr2=0x7fd834069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:56.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.979+0000 7fd8575dd640 1 --2- 172.21.15.122:0/2329617448 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd834066e40 0x7fd834069300 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd84c007fd0 tx=0x7fd84c009040 comp rx=0 tx=0).stop 2024-03-16T20:45:56.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.979+0000 7fd8575dd640 1 -- 172.21.15.122:0/2329617448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8501dff10 msgr2=0x7fd8501e0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:56.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.979+0000 7fd8575dd640 1 --2- 172.21.15.122:0/2329617448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8501dff10 0x7fd8501e0390 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd84801f8d0 tx=0x7fd83c005800 comp rx=0 tx=0).stop 2024-03-16T20:45:56.985 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.979+0000 7fd8575dd640 1 -- 172.21.15.122:0/2329617448 shutdown_connections 2024-03-16T20:45:56.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.979+0000 7fd8575dd640 1 --2- 172.21.15.122:0/2329617448 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd834066e40 0x7fd834069300 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.979+0000 7fd8575dd640 1 --2- 172.21.15.122:0/2329617448 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd8501dff10 0x7fd8501e0390 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.979+0000 7fd8575dd640 1 --2- 172.21.15.122:0/2329617448 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd8501dc9a0 0x7fd8501df9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.979+0000 7fd8575dd640 1 --2- 172.21.15.122:0/2329617448 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd85006d350 0x7fd8501dc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:56.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.979+0000 7fd8575dd640 1 -- 172.21.15.122:0/2329617448 >> 172.21.15.122:0/2329617448 conn(0x7fd85006fc00 msgr2=0x7fd8500bb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:56.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.979+0000 7fd8575dd640 1 -- 172.21.15.122:0/2329617448 shutdown_connections 2024-03-16T20:45:56.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:56.979+0000 7fd8575dd640 1 -- 172.21.15.122:0/2329617448 wait complete. 2024-03-16T20:45:57.001 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T20:45:57.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.115+0000 7f6df32c5640 1 Processor -- start 2024-03-16T20:45:57.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.115+0000 7f6df32c5640 1 -- start start 2024-03-16T20:45:57.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.115+0000 7f6df32c5640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f6dec06b1f0 0x7f6dec0b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.115+0000 7f6df32c5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dec06f6f0 0x7f6dec141270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.115+0000 7f6df32c5640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6dec1417b0 0x7f6dec14a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.115+0000 7f6df32c5640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dec147ca0 con 0x7f6dec1417b0 2024-03-16T20:45:57.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.115+0000 7f6df32c5640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dec06b5d0 con 0x7f6dec06b1f0 2024-03-16T20:45:57.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.115+0000 7f6df32c5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec147650 con 0x7f6dec06f6f0 2024-03-16T20:45:57.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.119+0000 7f6df1862640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6dec1417b0 0x7f6dec14a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.119+0000 7f6df1862640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f6dec1417b0 0x7f6dec14a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49300/0 (socket says 172.21.15.122:49300) 2024-03-16T20:45:57.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.119+0000 7f6df1862640 1 -- 172.21.15.122:0/2792578131 learned_addr learned my addr 172.21.15.122:0/2792578131 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:57.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.119+0000 7f6df1061640 1 --2- 172.21.15.122:0/2792578131 >> v2:172.21.15.195:3300/0 conn(0x7f6dec06b1f0 0x7f6dec0b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.119+0000 7f6df0860640 1 --2- 172.21.15.122:0/2792578131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dec06f6f0 0x7f6dec141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.119+0000 7f6df1862640 1 -- 172.21.15.122:0/2792578131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dec06f6f0 msgr2=0x7f6dec141270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:57.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.119+0000 7f6df1862640 1 --2- 172.21.15.122:0/2792578131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dec06f6f0 0x7f6dec141270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.119+0000 7f6df1862640 1 -- 172.21.15.122:0/2792578131 >> v2:172.21.15.195:3300/0 conn(0x7f6dec06b1f0 msgr2=0x7f6dec0b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:57.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.119+0000 7f6df1862640 1 --2- 172.21.15.122:0/2792578131 >> v2:172.21.15.195:3300/0 conn(0x7f6dec06b1f0 0x7f6dec0b7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.119+0000 7f6df1862640 1 -- 172.21.15.122:0/2792578131 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dec147930 con 0x7f6dec1417b0 2024-03-16T20:45:57.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df1862640 1 --2- 172.21.15.122:0/2792578131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dec1417b0 0x7f6dec14a870 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6dec067cd0 tx=0x7f6de401c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b58989e9db68d33 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:57.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6de3fff640 1 -- 172.21.15.122:0/2792578131 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de40a64f0 con 0x7f6dec1417b0 2024-03-16T20:45:57.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6de3fff640 1 -- 172.21.15.122:0/2792578131 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de40a6650 con 0x7f6dec1417b0 2024-03-16T20:45:57.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6de3fff640 1 -- 172.21.15.122:0/2792578131 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de40a69e0 con 0x7f6dec1417b0 2024-03-16T20:45:57.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 -- 172.21.15.122:0/2792578131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dec1417b0 msgr2=0x7f6dec14a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:57.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 --2- 172.21.15.122:0/2792578131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dec1417b0 0x7f6dec14a870 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6dec067cd0 tx=0x7f6de401c6d0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 -- 172.21.15.122:0/2792578131 shutdown_connections 2024-03-16T20:45:57.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 --2- 172.21.15.122:0/2792578131 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dec1417b0 0x7f6dec14a870 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 --2- 172.21.15.122:0/2792578131 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dec06f6f0 0x7f6dec141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 --2- 172.21.15.122:0/2792578131 >> v2:172.21.15.195:3300/0 conn(0x7f6dec06b1f0 0x7f6dec0b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 -- 172.21.15.122:0/2792578131 >> 172.21.15.122:0/2792578131 conn(0x7f6dec070d30 msgr2=0x7f6dec071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:57.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 -- 172.21.15.122:0/2792578131 shutdown_connections 2024-03-16T20:45:57.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 -- 172.21.15.122:0/2792578131 wait complete. 2024-03-16T20:45:57.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 Processor -- start 2024-03-16T20:45:57.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 -- start start 2024-03-16T20:45:57.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dec06b1f0 0x7f6dec1433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dec06f6f0 0x7f6dec139840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dec1417b0 0x7f6dec139d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec142710 con 0x7f6dec1417b0 2024-03-16T20:45:57.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec1445f0 con 0x7f6dec06b1f0 2024-03-16T20:45:57.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dec144030 con 0x7f6dec06f6f0 2024-03-16T20:45:57.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df1862640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dec1417b0 0x7f6dec139d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df1862640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dec1417b0 0x7f6dec139d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49306/0 (socket says 172.21.15.122:49306) 2024-03-16T20:45:57.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df1862640 1 -- 172.21.15.122:0/1821862648 learned_addr learned my addr 172.21.15.122:0/1821862648 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:57.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df1061640 1 --2- 172.21.15.122:0/1821862648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dec06b1f0 0x7f6dec1433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df0860640 1 --2- 172.21.15.122:0/1821862648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dec06f6f0 0x7f6dec139840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df1862640 1 -- 172.21.15.122:0/1821862648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dec06f6f0 msgr2=0x7f6dec139840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:57.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df1862640 1 --2- 172.21.15.122:0/1821862648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dec06f6f0 0x7f6dec139840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df1862640 1 -- 172.21.15.122:0/1821862648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dec06b1f0 msgr2=0x7f6dec1433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:57.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df1862640 1 --2- 172.21.15.122:0/1821862648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dec06b1f0 0x7f6dec1433f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df1862640 1 -- 172.21.15.122:0/1821862648 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de4009b30 con 0x7f6dec1417b0 2024-03-16T20:45:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df0860640 1 --2- 172.21.15.122:0/1821862648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dec06f6f0 0x7f6dec139840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df1862640 1 --2- 172.21.15.122:0/1821862648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dec1417b0 0x7f6dec139d80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009040 tx=0x7f6de401af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6de1ffb640 1 -- 172.21.15.122:0/1821862648 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de40b4040 con 0x7f6dec1417b0 2024-03-16T20:45:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6de1ffb640 1 -- 172.21.15.122:0/1821862648 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6de40a6bd0 con 0x7f6dec1417b0 2024-03-16T20:45:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 -- 172.21.15.122:0/1821862648 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dec144310 con 0x7f6dec1417b0 2024-03-16T20:45:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6de1ffb640 1 -- 172.21.15.122:0/1821862648 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6de40af480 con 0x7f6dec1417b0 2024-03-16T20:45:57.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.123+0000 7f6df32c5640 1 -- 172.21.15.122:0/1821862648 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dec143cb0 con 0x7f6dec1417b0 2024-03-16T20:45:57.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.127+0000 7f6df32c5640 1 -- 172.21.15.122:0/1821862648 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dec144170 con 0x7f6dec1417b0 2024-03-16T20:45:57.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.127+0000 7f6de1ffb640 1 -- 172.21.15.122:0/1821862648 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f6de40af5e0 con 0x7f6dec1417b0 2024-03-16T20:45:57.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.127+0000 7f6de1ffb640 1 --2- 172.21.15.122:0/1821862648 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6dd0066e40 0x7f6dd0069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.127+0000 7f6de1ffb640 1 -- 172.21.15.122:0/1821862648 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6de40aa070 con 0x7f6dec1417b0 2024-03-16T20:45:57.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.127+0000 7f6df1061640 1 --2- 172.21.15.122:0/1821862648 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6dd0066e40 0x7f6dd0069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.133 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.127+0000 7f6df1061640 1 --2- 172.21.15.122:0/1821862648 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6dd0066e40 0x7f6dd0069300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6de800b930 tx=0x7f6de8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:57.135 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.131+0000 7f6de1ffb640 1 -- 172.21.15.122:0/1821862648 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f6dec144170 con 0x7f6dec1417b0 2024-03-16T20:45:57.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.283+0000 7f6df32c5640 1 -- 172.21.15.122:0/1821862648 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6dec142880 con 0x7f6dec1417b0 2024-03-16T20:45:57.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.283+0000 7f6de1ffb640 1 -- 172.21.15.122:0/1821862648 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6dec142880 con 0x7f6dec1417b0 2024-03-16T20:45:57.290 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-03-16T20:45:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.287+0000 7f6df32c5640 1 -- 172.21.15.122:0/1821862648 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6dd0066e40 msgr2=0x7f6dd0069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.287+0000 7f6df32c5640 1 --2- 172.21.15.122:0/1821862648 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6dd0066e40 0x7f6dd0069300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6de800b930 tx=0x7f6de8008040 comp rx=0 tx=0).stop 2024-03-16T20:45:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.287+0000 7f6df32c5640 1 -- 172.21.15.122:0/1821862648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dec1417b0 msgr2=0x7f6dec139d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.287+0000 7f6df32c5640 1 --2- 172.21.15.122:0/1821862648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dec1417b0 0x7f6dec139d80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6de4009040 tx=0x7f6de401af70 comp rx=0 tx=0).stop 2024-03-16T20:45:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.287+0000 7f6df32c5640 1 -- 172.21.15.122:0/1821862648 shutdown_connections 2024-03-16T20:45:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.287+0000 7f6df32c5640 1 --2- 172.21.15.122:0/1821862648 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f6dd0066e40 0x7f6dd0069300 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.287+0000 7f6df32c5640 1 --2- 172.21.15.122:0/1821862648 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6dec1417b0 0x7f6dec139d80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.287+0000 7f6df32c5640 1 --2- 172.21.15.122:0/1821862648 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f6dec06f6f0 0x7f6dec139840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.287+0000 7f6df32c5640 1 --2- 172.21.15.122:0/1821862648 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f6dec06b1f0 0x7f6dec1433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.287+0000 7f6df32c5640 1 -- 172.21.15.122:0/1821862648 >> 172.21.15.122:0/1821862648 conn(0x7f6dec070d30 msgr2=0x7f6dec0b8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.287+0000 7f6df32c5640 1 -- 172.21.15.122:0/1821862648 shutdown_connections 2024-03-16T20:45:57.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.287+0000 7f6df32c5640 1 -- 172.21.15.122:0/1821862648 wait complete. 2024-03-16T20:45:57.307 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T20:45:57.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.423+0000 7fd2a7941640 1 Processor -- start 2024-03-16T20:45:57.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd2a7941640 1 -- start start 2024-03-16T20:45:57.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd2a7941640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2a00b7d40 0x7fd2a00b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd2a7941640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd2a0148480 0x7fd2a014f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd2a7941640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a00b6740 con 0x7fd2a00b7540 2024-03-16T20:45:57.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd2a7941640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a00b23d0 con 0x7fd2a0148480 2024-03-16T20:45:57.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd2a7941640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a00b7110 con 0x7fd2a00b7d40 2024-03-16T20:45:57.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd2a56dd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fd2a00b7540 0x7fd2a013ef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:37492/0 (socket says 172.21.15.122:37492) 2024-03-16T20:45:57.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd2a56dd640 1 -- 172.21.15.122:0/1865680453 learned_addr learned my addr 172.21.15.122:0/1865680453 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:57.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd297fff640 1 -- 172.21.15.122:0/1865680453 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862824544 0 0) 0x7fd2a00b6740 con 0x7fd2a00b7540 2024-03-16T20:45:57.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd2a4edc640 1 --2- 172.21.15.122:0/1865680453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2a00b7d40 0x7fd2a00b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd297fff640 1 -- 172.21.15.122:0/1865680453 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd28c003530 con 0x7fd2a00b7540 2024-03-16T20:45:57.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd2a5ede640 1 --2- 172.21.15.122:0/1865680453 >> v2:172.21.15.195:3300/0 conn(0x7fd2a0148480 0x7fd2a014f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd297fff640 1 -- 172.21.15.122:0/1865680453 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4183709773 0 0) 0x7fd28c003530 con 0x7fd2a00b7540 2024-03-16T20:45:57.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd297fff640 1 -- 172.21.15.122:0/1865680453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2a00b7d40 msgr2=0x7fd2a00b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:57.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd297fff640 1 --2- 172.21.15.122:0/1865680453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2a00b7d40 0x7fd2a00b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd297fff640 1 -- 172.21.15.122:0/1865680453 >> v2:172.21.15.195:3300/0 conn(0x7fd2a0148480 msgr2=0x7fd2a014f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:57.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd297fff640 1 --2- 172.21.15.122:0/1865680453 >> v2:172.21.15.195:3300/0 conn(0x7fd2a0148480 0x7fd2a014f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd297fff640 1 -- 172.21.15.122:0/1865680453 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a00b7920 con 0x7fd2a00b7540 2024-03-16T20:45:57.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd297fff640 1 -- 172.21.15.122:0/1865680453 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7fd298001a00 con 0x7fd2a00b7540 2024-03-16T20:45:57.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd297fff640 1 -- 172.21.15.122:0/1865680453 >> v1:172.21.15.122:6789/0 conn(0x7fd2a00b7540 legacy=0x7fd2a013ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:57.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd297fff640 1 --2- 172.21.15.122:0/1865680453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c0039b0 0x7fd28c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd297fff640 1 --2- 172.21.15.122:0/1865680453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd28c0047d0 0x7fd28c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd297fff640 1 --2- 172.21.15.122:0/1865680453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2a00b7d40 0x7fd28c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd297fff640 1 -- 172.21.15.122:0/1865680453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a00b23d0 con 0x7fd28c0039b0 2024-03-16T20:45:57.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd297fff640 1 -- 172.21.15.122:0/1865680453 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a00b7110 con 0x7fd28c0047d0 2024-03-16T20:45:57.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd297fff640 1 -- 172.21.15.122:0/1865680453 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28c00ba70 con 0x7fd2a00b7d40 2024-03-16T20:45:57.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd2a56dd640 1 --2- 172.21.15.122:0/1865680453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c0039b0 0x7fd28c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd2a4edc640 1 --2- 172.21.15.122:0/1865680453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd28c0047d0 0x7fd28c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd2a56dd640 1 -- 172.21.15.122:0/1865680453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2a00b7d40 msgr2=0x7fd28c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:57.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd2a56dd640 1 --2- 172.21.15.122:0/1865680453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2a00b7d40 0x7fd28c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd2a56dd640 1 -- 172.21.15.122:0/1865680453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd28c0047d0 msgr2=0x7fd28c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:57.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd2a56dd640 1 --2- 172.21.15.122:0/1865680453 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd28c0047d0 0x7fd28c004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.427+0000 7fd2a56dd640 1 -- 172.21.15.122:0/1865680453 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c003530 con 0x7fd28c0039b0 2024-03-16T20:45:57.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a56dd640 1 --2- 172.21.15.122:0/1865680453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c0039b0 0x7fd28c003dd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd29800a7f0 tx=0x7fd29801d270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8efc976d96832c5d server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:57.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd297fff640 1 -- 172.21.15.122:0/1865680453 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2980072d0 con 0x7fd28c0039b0 2024-03-16T20:45:57.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd297fff640 1 -- 172.21.15.122:0/1865680453 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2980a8070 con 0x7fd28c0039b0 2024-03-16T20:45:57.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd297fff640 1 -- 172.21.15.122:0/1865680453 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd298007690 con 0x7fd28c0039b0 2024-03-16T20:45:57.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a7941640 1 -- 172.21.15.122:0/1865680453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c0039b0 msgr2=0x7fd28c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:57.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a7941640 1 --2- 172.21.15.122:0/1865680453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c0039b0 0x7fd28c003dd0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd29800a7f0 tx=0x7fd29801d270 comp rx=0 tx=0).stop 2024-03-16T20:45:57.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a5ede640 1 -- 172.21.15.122:0/1865680453 reap_dead start 2024-03-16T20:45:57.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a7941640 1 -- 172.21.15.122:0/1865680453 shutdown_connections 2024-03-16T20:45:57.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a7941640 1 -- 172.21.15.122:0/1865680453 >> 172.21.15.122:0/1865680453 conn(0x7fd2a006fc00 msgr2=0x7fd2a0070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:57.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a7941640 1 -- 172.21.15.122:0/1865680453 shutdown_connections 2024-03-16T20:45:57.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a7941640 1 -- 172.21.15.122:0/1865680453 wait complete. 2024-03-16T20:45:57.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a7941640 1 Processor -- start 2024-03-16T20:45:57.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a7941640 1 -- start start 2024-03-16T20:45:57.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a7941640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd28c0039b0 0x7fd2a0144130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a7941640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2a00b95d0 0x7fd2a014d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a7941640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2a0135160 0x7fd2a01356d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a7941640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a00b8b10 con 0x7fd2a0135160 2024-03-16T20:45:57.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a7941640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a00b8d00 con 0x7fd28c0039b0 2024-03-16T20:45:57.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a7941640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a00b9b00 con 0x7fd2a00b95d0 2024-03-16T20:45:57.440 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a5ede640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2a0135160 0x7fd2a01356d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a5ede640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2a0135160 0x7fd2a01356d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49314/0 (socket says 172.21.15.122:49314) 2024-03-16T20:45:57.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a5ede640 1 -- 172.21.15.122:0/3296869680 learned_addr learned my addr 172.21.15.122:0/3296869680 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:57.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a4edc640 1 --2- 172.21.15.122:0/3296869680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2a00b95d0 0x7fd2a014d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.431+0000 7fd2a56dd640 1 --2- 172.21.15.122:0/3296869680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd28c0039b0 0x7fd2a0144130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd2a5ede640 1 -- 172.21.15.122:0/3296869680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2a00b95d0 msgr2=0x7fd2a014d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:57.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd2a5ede640 1 --2- 172.21.15.122:0/3296869680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2a00b95d0 0x7fd2a014d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd2a5ede640 1 -- 172.21.15.122:0/3296869680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd28c0039b0 msgr2=0x7fd2a0144130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:57.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd2a5ede640 1 --2- 172.21.15.122:0/3296869680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd28c0039b0 0x7fd2a0144130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd2a5ede640 1 -- 172.21.15.122:0/3296869680 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a00ba430 con 0x7fd2a0135160 2024-03-16T20:45:57.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd2a56dd640 1 --2- 172.21.15.122:0/3296869680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd28c0039b0 0x7fd2a0144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:57.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd2a5ede640 1 --2- 172.21.15.122:0/3296869680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2a0135160 0x7fd2a01356d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0074260 tx=0x7fd29c00d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:57.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd295ffb640 1 -- 172.21.15.122:0/3296869680 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c017070 con 0x7fd2a0135160 2024-03-16T20:45:57.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd295ffb640 1 -- 172.21.15.122:0/3296869680 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29c004020 con 0x7fd2a0135160 2024-03-16T20:45:57.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd295ffb640 1 -- 172.21.15.122:0/3296869680 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c0041b0 con 0x7fd2a0135160 2024-03-16T20:45:57.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd2a7941640 1 -- 172.21.15.122:0/3296869680 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a0134c40 con 0x7fd2a0135160 2024-03-16T20:45:57.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd2a7941640 1 -- 172.21.15.122:0/3296869680 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a0144be0 con 0x7fd2a0135160 2024-03-16T20:45:57.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd2a7941640 1 -- 172.21.15.122:0/3296869680 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a00ba270 con 0x7fd2a0135160 2024-03-16T20:45:57.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd295ffb640 1 -- 172.21.15.122:0/3296869680 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fd29c004310 con 0x7fd2a0135160 2024-03-16T20:45:57.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd295ffb640 1 --2- 172.21.15.122:0/3296869680 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd284066e40 0x7fd284069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd295ffb640 1 -- 172.21.15.122:0/3296869680 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd29c0841e0 con 0x7fd2a0135160 2024-03-16T20:45:57.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd2a56dd640 1 --2- 172.21.15.122:0/3296869680 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd284066e40 0x7fd284069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.435+0000 7fd2a56dd640 1 --2- 172.21.15.122:0/3296869680 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd284066e40 0x7fd284069300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd2980ad000 tx=0x7fd298009050 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:57.445 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.439+0000 7fd295ffb640 1 -- 172.21.15.122:0/3296869680 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd29c050f40 con 0x7fd2a0135160 2024-03-16T20:45:57.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.587+0000 7fd2a7941640 1 -- 172.21.15.122:0/3296869680 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd2a00b2db0 con 0x7fd2a0135160 2024-03-16T20:45:57.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.587+0000 7fd295ffb640 1 -- 172.21.15.122:0/3296869680 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd29c054600 con 0x7fd2a0135160 2024-03-16T20:45:57.594 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-03-16T20:45:57.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.591+0000 7fd2a7941640 1 -- 172.21.15.122:0/3296869680 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd284066e40 msgr2=0x7fd284069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:57.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.591+0000 7fd2a7941640 1 --2- 172.21.15.122:0/3296869680 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd284066e40 0x7fd284069300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd2980ad000 tx=0x7fd298009050 comp rx=0 tx=0).stop 2024-03-16T20:45:57.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.591+0000 7fd2a7941640 1 -- 172.21.15.122:0/3296869680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2a0135160 msgr2=0x7fd2a01356d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:57.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.591+0000 7fd2a7941640 1 --2- 172.21.15.122:0/3296869680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2a0135160 0x7fd2a01356d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0074260 tx=0x7fd29c00d820 comp rx=0 tx=0).stop 2024-03-16T20:45:57.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.591+0000 7fd2a7941640 1 -- 172.21.15.122:0/3296869680 shutdown_connections 2024-03-16T20:45:57.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.591+0000 7fd2a7941640 1 --2- 172.21.15.122:0/3296869680 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd284066e40 0x7fd284069300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.591+0000 7fd2a7941640 1 --2- 172.21.15.122:0/3296869680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd2a0135160 0x7fd2a01356d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.591+0000 7fd2a7941640 1 --2- 172.21.15.122:0/3296869680 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2a00b95d0 0x7fd2a014d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.591+0000 7fd2a7941640 1 --2- 172.21.15.122:0/3296869680 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd28c0039b0 0x7fd2a0144130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.591+0000 7fd2a7941640 1 -- 172.21.15.122:0/3296869680 >> 172.21.15.122:0/3296869680 conn(0x7fd2a006fc00 msgr2=0x7fd2a00b55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:57.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.591+0000 7fd2a7941640 1 -- 172.21.15.122:0/3296869680 shutdown_connections 2024-03-16T20:45:57.597 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.591+0000 7fd2a7941640 1 -- 172.21.15.122:0/3296869680 wait complete. 2024-03-16T20:45:57.612 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-16T20:45:57.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.711+0000 7f2dd59eb640 1 Processor -- start 2024-03-16T20:45:57.716 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.711+0000 7f2dd59eb640 1 -- start start 2024-03-16T20:45:57.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.711+0000 7f2dd59eb640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2dd00b90c0 0x7f2dd006ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.711+0000 7f2dd59eb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dd0144c00 0x7f2dd00b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.711+0000 7f2dd59eb640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd006daf0 con 0x7f2dd0144290 2024-03-16T20:45:57.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.711+0000 7f2dd59eb640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd00b1390 con 0x7f2dd00b90c0 2024-03-16T20:45:57.717 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.711+0000 7f2dd59eb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd006dd40 con 0x7f2dd0144c00 2024-03-16T20:45:57.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.711+0000 7f2dcf7fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dd0144c00 0x7f2dd00b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.711+0000 7f2dce7fc640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2dd00b90c0 0x7f2dd006ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.711+0000 7f2dcf7fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dd0144c00 0x7f2dd00b7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:52042/0 (socket says 172.21.15.122:52042) 2024-03-16T20:45:57.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.711+0000 7f2dcf7fe640 1 -- 172.21.15.122:0/2549222803 learned_addr learned my addr 172.21.15.122:0/2549222803 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:57.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.711+0000 7f2dce7fc640 1 -- 172.21.15.122:0/2549222803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dd0144c00 msgr2=0x7f2dd00b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:57.718 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.711+0000 7f2dce7fc640 1 --2- 172.21.15.122:0/2549222803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dd0144c00 0x7f2dd00b7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.711+0000 7f2dce7fc640 1 -- 172.21.15.122:0/2549222803 >> v1:172.21.15.195:6789/0 conn(0x7f2dd0144290 legacy=0x7f2dd013df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:57.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.711+0000 7f2dce7fc640 1 -- 172.21.15.122:0/2549222803 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd013e690 con 0x7f2dd00b90c0 2024-03-16T20:45:57.719 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.711+0000 7f2dcf7fe640 1 --2- 172.21.15.122:0/2549222803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dd0144c00 0x7f2dd00b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:45:57.720 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.715+0000 7f2dce7fc640 1 --2- 172.21.15.122:0/2549222803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dd00b90c0 0x7f2dd006ed70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2db800dbe0 tx=0x7f2db8020690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=647792a5ba9cc3a3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:57.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.715+0000 7f2dcdffb640 1 -- 172.21.15.122:0/2549222803 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db800d2d0 con 0x7f2dd00b90c0 2024-03-16T20:45:57.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.715+0000 7f2dcdffb640 1 -- 172.21.15.122:0/2549222803 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db80ad480 con 0x7f2dd00b90c0 2024-03-16T20:45:57.721 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.715+0000 7f2dcdffb640 1 -- 172.21.15.122:0/2549222803 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db80ad810 con 0x7f2dd00b90c0 2024-03-16T20:45:57.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.715+0000 7f2dd59eb640 1 -- 172.21.15.122:0/2549222803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dd00b90c0 msgr2=0x7f2dd006ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:57.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.715+0000 7f2dd59eb640 1 --2- 172.21.15.122:0/2549222803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dd00b90c0 0x7f2dd006ed70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2db800dbe0 tx=0x7f2db8020690 comp rx=0 tx=0).stop 2024-03-16T20:45:57.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.715+0000 7f2dd59eb640 1 -- 172.21.15.122:0/2549222803 shutdown_connections 2024-03-16T20:45:57.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.715+0000 7f2dd59eb640 1 --2- 172.21.15.122:0/2549222803 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dd0144c00 0x7f2dd00b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.715+0000 7f2dd59eb640 1 --2- 172.21.15.122:0/2549222803 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dd00b90c0 0x7f2dd006ed70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.715+0000 7f2dd59eb640 1 -- 172.21.15.122:0/2549222803 >> 172.21.15.122:0/2549222803 conn(0x7f2dd0070d30 msgr2=0x7f2dd0071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:57.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.715+0000 7f2dd59eb640 1 -- 172.21.15.122:0/2549222803 shutdown_connections 2024-03-16T20:45:57.722 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.715+0000 7f2dd59eb640 1 -- 172.21.15.122:0/2549222803 wait complete. 2024-03-16T20:45:57.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dd59eb640 1 Processor -- start 2024-03-16T20:45:57.723 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dd59eb640 1 -- start start 2024-03-16T20:45:57.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dd59eb640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dd00b90c0 0x7f2dd01469b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dd59eb640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dd0144290 0x7f2dd00b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dd59eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd0144c00 0x7f2dd00b2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dd59eb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd00b1a70 con 0x7f2dd0144c00 2024-03-16T20:45:57.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dd59eb640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd0137a40 con 0x7f2dd0144290 2024-03-16T20:45:57.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dd59eb640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd01377a0 con 0x7f2dd00b90c0 2024-03-16T20:45:57.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dcf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd0144c00 0x7f2dd00b2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.725 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dcf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd0144c00 0x7f2dd00b2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49320/0 (socket says 172.21.15.122:49320) 2024-03-16T20:45:57.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dcf7fe640 1 -- 172.21.15.122:0/2450117253 learned_addr learned my addr 172.21.15.122:0/2450117253 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:57.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dce7fc640 1 --2- 172.21.15.122:0/2450117253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dd0144290 0x7f2dd00b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dceffd640 1 --2- 172.21.15.122:0/2450117253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dd00b90c0 0x7f2dd01469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dcf7fe640 1 -- 172.21.15.122:0/2450117253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dd00b90c0 msgr2=0x7f2dd01469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:57.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dcf7fe640 1 --2- 172.21.15.122:0/2450117253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dd00b90c0 0x7f2dd01469b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dcf7fe640 1 -- 172.21.15.122:0/2450117253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dd0144290 msgr2=0x7f2dd00b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:57.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dcf7fe640 1 --2- 172.21.15.122:0/2450117253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dd0144290 0x7f2dd00b1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dcf7fe640 1 -- 172.21.15.122:0/2450117253 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc800bb50 con 0x7f2dd0144c00 2024-03-16T20:45:57.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dcf7fe640 1 --2- 172.21.15.122:0/2450117253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd0144c00 0x7f2dd00b2400 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009f90 tx=0x7f2dc80085d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:57.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dc7fff640 1 -- 172.21.15.122:0/2450117253 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc8010040 con 0x7f2dd0144c00 2024-03-16T20:45:57.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dc7fff640 1 -- 172.21.15.122:0/2450117253 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc8003ba0 con 0x7f2dd0144c00 2024-03-16T20:45:57.726 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dd59eb640 1 -- 172.21.15.122:0/2450117253 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2db800d8c0 con 0x7f2dd0144c00 2024-03-16T20:45:57.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dc7fff640 1 -- 172.21.15.122:0/2450117253 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc800bdb0 con 0x7f2dd0144c00 2024-03-16T20:45:57.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dd59eb640 1 -- 172.21.15.122:0/2450117253 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd00b33e0 con 0x7f2dd0144c00 2024-03-16T20:45:57.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dd59eb640 1 -- 172.21.15.122:0/2450117253 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd00b2940 con 0x7f2dd0144c00 2024-03-16T20:45:57.727 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.719+0000 7f2dc7fff640 1 -- 172.21.15.122:0/2450117253 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f2dc8003f10 con 0x7f2dd0144c00 2024-03-16T20:45:57.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.723+0000 7f2dc7fff640 1 --2- 172.21.15.122:0/2450117253 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2db0066e70 0x7f2db0069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:57.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.723+0000 7f2dc7fff640 1 -- 172.21.15.122:0/2450117253 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2dc8085130 con 0x7f2dd0144c00 2024-03-16T20:45:57.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.723+0000 7f2dceffd640 1 --2- 172.21.15.122:0/2450117253 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2db0066e70 0x7f2db0069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:57.728 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.723+0000 7f2dceffd640 1 --2- 172.21.15.122:0/2450117253 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2db0066e70 0x7f2db0069330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0001770 tx=0x7f2dc0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:57.735 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.731+0000 7f2dc7fff640 1 -- 172.21.15.122:0/2450117253 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2dc8052060 con 0x7f2dd0144c00 2024-03-16T20:45:57.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.895+0000 7f2dd59eb640 1 -- 172.21.15.122:0/2450117253 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2dd00ba6b0 con 0x7f2dd0144c00 2024-03-16T20:45:57.902 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.899+0000 7f2dc7fff640 1 -- 172.21.15.122:0/2450117253 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v30) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2dc8055720 con 0x7f2dd0144c00 2024-03-16T20:45:57.902 INFO:teuthology.orchestra.run.smithi122.stderr:listed 0 entries 2024-03-16T20:45:57.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.899+0000 7f2dd59eb640 1 -- 172.21.15.122:0/2450117253 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2db0066e70 msgr2=0x7f2db0069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:57.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.899+0000 7f2dd59eb640 1 --2- 172.21.15.122:0/2450117253 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2db0066e70 0x7f2db0069330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0001770 tx=0x7f2dc0009040 comp rx=0 tx=0).stop 2024-03-16T20:45:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.899+0000 7f2dd59eb640 1 -- 172.21.15.122:0/2450117253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd0144c00 msgr2=0x7f2dd00b2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.899+0000 7f2dd59eb640 1 --2- 172.21.15.122:0/2450117253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd0144c00 0x7f2dd00b2400 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009f90 tx=0x7f2dc80085d0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.899+0000 7f2dd59eb640 1 -- 172.21.15.122:0/2450117253 shutdown_connections 2024-03-16T20:45:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.899+0000 7f2dd59eb640 1 --2- 172.21.15.122:0/2450117253 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2db0066e70 0x7f2db0069330 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.899+0000 7f2dd59eb640 1 --2- 172.21.15.122:0/2450117253 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dd0144c00 0x7f2dd00b2400 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.899+0000 7f2dd59eb640 1 --2- 172.21.15.122:0/2450117253 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dd0144290 0x7f2dd00b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.899+0000 7f2dd59eb640 1 --2- 172.21.15.122:0/2450117253 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dd00b90c0 0x7f2dd01469b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.899+0000 7f2dd59eb640 1 -- 172.21.15.122:0/2450117253 >> 172.21.15.122:0/2450117253 conn(0x7f2dd0070d30 msgr2=0x7f2dd0139960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.899+0000 7f2dd59eb640 1 -- 172.21.15.122:0/2450117253 shutdown_connections 2024-03-16T20:45:57.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:57.899+0000 7f2dd59eb640 1 -- 172.21.15.122:0/2450117253 wait complete. 2024-03-16T20:45:57.919 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi122.front.sepia.ceph.com... 2024-03-16T20:45:57.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:57.919 DEBUG:teuthology.orchestra.run.smithi122:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-03-16T20:45:57.929 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi122.stderr:ceph-fuse[36767]: fuse finished with error 0 and tester_r 0 2024-03-16T20:45:57.930 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi122.front.sepia.ceph.com 2024-03-16T20:45:57.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:45:57.931 DEBUG:teuthology.orchestra.run.smithi122:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-16T20:45:58.093 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T20:45:58.249 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.243+0000 7f042acc1640 1 Processor -- start 2024-03-16T20:45:58.250 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.243+0000 7f042acc1640 1 -- start start 2024-03-16T20:45:58.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f042acc1640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f04240b64c0 0x7f04241487f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f042acc1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0424148d30 0x7f042413ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f042acc1640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04240b1760 con 0x7f042413f220 2024-03-16T20:45:58.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f042acc1640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04240b19b0 con 0x7f04240b64c0 2024-03-16T20:45:58.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f042acc1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04240b3140 con 0x7f0424148d30 2024-03-16T20:45:58.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f0428a5d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f04240b64c0 0x7f04241487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.251 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f042925e640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f042413f220 0x7f0424144f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:37498/0 (socket says 172.21.15.122:37498) 2024-03-16T20:45:58.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f042925e640 1 -- 172.21.15.122:0/3191537873 learned_addr learned my addr 172.21.15.122:0/3191537873 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:58.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f0428a5d640 1 -- 172.21.15.122:0/3191537873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0424148d30 msgr2=0x7f042413ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:58.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f041bfff640 1 --2- 172.21.15.122:0/3191537873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0424148d30 0x7f042413ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f0428a5d640 1 --2- 172.21.15.122:0/3191537873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0424148d30 0x7f042413ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f0428a5d640 1 -- 172.21.15.122:0/3191537873 >> v1:172.21.15.122:6789/0 conn(0x7f042413f220 legacy=0x7f0424144f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:58.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f0428a5d640 1 -- 172.21.15.122:0/3191537873 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04240b2ed0 con 0x7f04240b64c0 2024-03-16T20:45:58.252 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f041bfff640 1 --2- 172.21.15.122:0/3191537873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0424148d30 0x7f042413ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:58.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f0428a5d640 1 --2- 172.21.15.122:0/3191537873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04240b64c0 0x7f04241487f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f04200097c0 tx=0x7f042001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ee4d657c7dfc19 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:58.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f041b7fe640 1 -- 172.21.15.122:0/3191537873 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0424145e30 con 0x7f04240b64c0 2024-03-16T20:45:58.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f041b7fe640 1 -- 172.21.15.122:0/3191537873 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04200a7050 con 0x7f04240b64c0 2024-03-16T20:45:58.253 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f041b7fe640 1 -- 172.21.15.122:0/3191537873 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04200ab820 con 0x7f04240b64c0 2024-03-16T20:45:58.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f042acc1640 1 -- 172.21.15.122:0/3191537873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04240b64c0 msgr2=0x7f04241487f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:58.254 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f042acc1640 1 --2- 172.21.15.122:0/3191537873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04240b64c0 0x7f04241487f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f04200097c0 tx=0x7f042001c380 comp rx=0 tx=0).stop 2024-03-16T20:45:58.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f042acc1640 1 -- 172.21.15.122:0/3191537873 shutdown_connections 2024-03-16T20:45:58.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f042acc1640 1 --2- 172.21.15.122:0/3191537873 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f0424148d30 0x7f042413ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f042acc1640 1 --2- 172.21.15.122:0/3191537873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f04240b64c0 0x7f04241487f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f042acc1640 1 -- 172.21.15.122:0/3191537873 >> 172.21.15.122:0/3191537873 conn(0x7f04240b7350 msgr2=0x7f04240b7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:58.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f042acc1640 1 -- 172.21.15.122:0/3191537873 shutdown_connections 2024-03-16T20:45:58.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.247+0000 7f042acc1640 1 -- 172.21.15.122:0/3191537873 wait complete. 2024-03-16T20:45:58.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042acc1640 1 Processor -- start 2024-03-16T20:45:58.255 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042acc1640 1 -- start start 2024-03-16T20:45:58.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042acc1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04240b64c0 0x7f042413c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042acc1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f042413f220 0x7f042413b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042acc1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0424148d30 0x7f04241e30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042acc1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042413cb10 con 0x7f0424148d30 2024-03-16T20:45:58.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042acc1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0424147bc0 con 0x7f04240b64c0 2024-03-16T20:45:58.258 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042acc1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f042413c820 con 0x7f042413f220 2024-03-16T20:45:58.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042925e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0424148d30 0x7f04241e30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042925e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0424148d30 0x7f04241e30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49346/0 (socket says 172.21.15.122:49346) 2024-03-16T20:45:58.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042925e640 1 -- 172.21.15.122:0/3938658887 learned_addr learned my addr 172.21.15.122:0/3938658887 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:58.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f041bfff640 1 --2- 172.21.15.122:0/3938658887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f042413f220 0x7f042413b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042925e640 1 -- 172.21.15.122:0/3938658887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f042413f220 msgr2=0x7f042413b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:58.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f0428a5d640 1 --2- 172.21.15.122:0/3938658887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04240b64c0 0x7f042413c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042925e640 1 --2- 172.21.15.122:0/3938658887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f042413f220 0x7f042413b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042925e640 1 -- 172.21.15.122:0/3938658887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04240b64c0 msgr2=0x7f042413c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:58.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042925e640 1 --2- 172.21.15.122:0/3938658887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04240b64c0 0x7f042413c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042925e640 1 -- 172.21.15.122:0/3938658887 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0420009420 con 0x7f0424148d30 2024-03-16T20:45:58.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042925e640 1 --2- 172.21.15.122:0/3938658887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0424148d30 0x7f04241e30a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f041c00cbc0 tx=0x7f041c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:58.259 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f04197fa640 1 -- 172.21.15.122:0/3938658887 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c00ad40 con 0x7f0424148d30 2024-03-16T20:45:58.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f04197fa640 1 -- 172.21.15.122:0/3938658887 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f041c012400 con 0x7f0424148d30 2024-03-16T20:45:58.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042acc1640 1 -- 172.21.15.122:0/3938658887 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0424146070 con 0x7f0424148d30 2024-03-16T20:45:58.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f04197fa640 1 -- 172.21.15.122:0/3938658887 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c00aea0 con 0x7f0424148d30 2024-03-16T20:45:58.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042acc1640 1 -- 172.21.15.122:0/3938658887 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042406c390 con 0x7f0424148d30 2024-03-16T20:45:58.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f042acc1640 1 -- 172.21.15.122:0/3938658887 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0424075280 con 0x7f0424148d30 2024-03-16T20:45:58.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f04197fa640 1 -- 172.21.15.122:0/3938658887 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f041c007070 con 0x7f0424148d30 2024-03-16T20:45:58.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.251+0000 7f04197fa640 1 --2- 172.21.15.122:0/3938658887 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0410066e40 0x7f0410069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.255+0000 7f04197fa640 1 -- 172.21.15.122:0/3938658887 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f041c0847d0 con 0x7f0424148d30 2024-03-16T20:45:58.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.255+0000 7f0428a5d640 1 --2- 172.21.15.122:0/3938658887 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0410066e40 0x7f0410069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.255+0000 7f0428a5d640 1 --2- 172.21.15.122:0/3938658887 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0410066e40 0x7f0410069300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f04200ad000 tx=0x7f0420008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:58.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.259+0000 7f04197fa640 1 -- 172.21.15.122:0/3938658887 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f041c051700 con 0x7f0424148d30 2024-03-16T20:45:58.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.447+0000 7f042acc1640 1 -- 172.21.15.122:0/3938658887 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0424149390 con 0x7f0424148d30 2024-03-16T20:45:58.453 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.447+0000 7f04197fa640 1 -- 172.21.15.122:0/3938658887 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+3401 (secure 0 0 0) 0x7f041c017020 con 0x7f0424148d30 2024-03-16T20:45:58.454 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:58.454 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3638578490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3638578490},{"type":"v1","addr":"172.21.15.195:6835","nonce":3638578490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T20:45:44.280433+0000","modified":"2024-03-16T20:45:46.325382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4995":{"gid":4995,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6833/1841055069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1841055069},{"type":"v1","addr":"172.21.15.122:6833","nonce":1841055069}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4995,"qdb_cluster":[4995]},"id":2}]} 2024-03-16T20:45:58.454 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 13 2024-03-16T20:45:58.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.451+0000 7f042acc1640 1 -- 172.21.15.122:0/3938658887 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0410066e40 msgr2=0x7f0410069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:58.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.451+0000 7f042acc1640 1 --2- 172.21.15.122:0/3938658887 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0410066e40 0x7f0410069300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f04200ad000 tx=0x7f0420008040 comp rx=0 tx=0).stop 2024-03-16T20:45:58.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.451+0000 7f042acc1640 1 -- 172.21.15.122:0/3938658887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0424148d30 msgr2=0x7f04241e30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:58.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.451+0000 7f042acc1640 1 --2- 172.21.15.122:0/3938658887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0424148d30 0x7f04241e30a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f041c00cbc0 tx=0x7f041c00a430 comp rx=0 tx=0).stop 2024-03-16T20:45:58.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.451+0000 7f042acc1640 1 -- 172.21.15.122:0/3938658887 shutdown_connections 2024-03-16T20:45:58.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.451+0000 7f042acc1640 1 --2- 172.21.15.122:0/3938658887 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0410066e40 0x7f0410069300 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.451+0000 7f042acc1640 1 --2- 172.21.15.122:0/3938658887 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0424148d30 0x7f04241e30a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.451+0000 7f042acc1640 1 --2- 172.21.15.122:0/3938658887 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f042413f220 0x7f042413b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.451+0000 7f042acc1640 1 --2- 172.21.15.122:0/3938658887 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f04240b64c0 0x7f042413c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.451+0000 7f042acc1640 1 -- 172.21.15.122:0/3938658887 >> 172.21.15.122:0/3938658887 conn(0x7f04240b7350 msgr2=0x7f0424074320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:58.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.451+0000 7f042acc1640 1 -- 172.21.15.122:0/3938658887 shutdown_connections 2024-03-16T20:45:58.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.451+0000 7f042acc1640 1 -- 172.21.15.122:0/3938658887 wait complete. 2024-03-16T20:45:58.478 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:45:58.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b42ed640 1 Processor -- start 2024-03-16T20:45:58.590 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b42ed640 1 -- start start 2024-03-16T20:45:58.591 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b42ed640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4ac066fa0 0x7fd4ac09a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b42ed640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd4ac0b5060 0x7fd4ac144f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b42ed640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ac099d90 con 0x7fd4ac072df0 2024-03-16T20:45:58.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b42ed640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ac1454b0 con 0x7fd4ac0b5060 2024-03-16T20:45:58.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b42ed640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac09bc00 con 0x7fd4ac066fa0 2024-03-16T20:45:58.593 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1888640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fd4ac072df0 0x7fd4ac13ee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:40486/0 (socket says 172.21.15.122:40486) 2024-03-16T20:45:58.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1888640 1 -- 172.21.15.122:0/384498906 learned_addr learned my addr 172.21.15.122:0/384498906 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:58.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b2089640 1 --2- 172.21.15.122:0/384498906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4ac066fa0 0x7fd4ac09a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b288a640 1 --2- 172.21.15.122:0/384498906 >> v2:172.21.15.195:3300/0 conn(0x7fd4ac0b5060 0x7fd4ac144f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1087640 1 -- 172.21.15.122:0/384498906 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008661799 0 0) 0x7fd4ac099d90 con 0x7fd4ac072df0 2024-03-16T20:45:58.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1087640 1 -- 172.21.15.122:0/384498906 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd498003530 con 0x7fd4ac072df0 2024-03-16T20:45:58.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1087640 1 -- 172.21.15.122:0/384498906 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1742756904 0 0) 0x7fd498003530 con 0x7fd4ac072df0 2024-03-16T20:45:58.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1087640 1 -- 172.21.15.122:0/384498906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4ac066fa0 msgr2=0x7fd4ac09a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:58.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1087640 1 --2- 172.21.15.122:0/384498906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4ac066fa0 0x7fd4ac09a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1087640 1 -- 172.21.15.122:0/384498906 >> v2:172.21.15.195:3300/0 conn(0x7fd4ac0b5060 msgr2=0x7fd4ac144f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:58.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1087640 1 --2- 172.21.15.122:0/384498906 >> v2:172.21.15.195:3300/0 conn(0x7fd4ac0b5060 0x7fd4ac144f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1087640 1 -- 172.21.15.122:0/384498906 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac067380 con 0x7fd4ac072df0 2024-03-16T20:45:58.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1087640 1 -- 172.21.15.122:0/384498906 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7fd49c001680 con 0x7fd4ac072df0 2024-03-16T20:45:58.594 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1087640 1 -- 172.21.15.122:0/384498906 >> v1:172.21.15.122:6789/0 conn(0x7fd4ac072df0 legacy=0x7fd4ac13ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:58.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1087640 1 --2- 172.21.15.122:0/384498906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd498003980 0x7fd498003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1087640 1 --2- 172.21.15.122:0/384498906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4980047d0 0x7fd498004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1087640 1 --2- 172.21.15.122:0/384498906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4ac066fa0 0x7fd498005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1087640 1 -- 172.21.15.122:0/384498906 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac1454b0 con 0x7fd498003980 2024-03-16T20:45:58.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1087640 1 -- 172.21.15.122:0/384498906 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac09bc00 con 0x7fd4980047d0 2024-03-16T20:45:58.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b1087640 1 -- 172.21.15.122:0/384498906 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49800b990 con 0x7fd4ac066fa0 2024-03-16T20:45:58.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b2089640 1 --2- 172.21.15.122:0/384498906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd498003980 0x7fd498003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b288a640 1 --2- 172.21.15.122:0/384498906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4ac066fa0 0x7fd498005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b2089640 1 -- 172.21.15.122:0/384498906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4ac066fa0 msgr2=0x7fd498005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:58.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b2089640 1 --2- 172.21.15.122:0/384498906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4ac066fa0 0x7fd498005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b2089640 1 -- 172.21.15.122:0/384498906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4980047d0 msgr2=0x7fd498004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:58.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b2089640 1 --2- 172.21.15.122:0/384498906 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4980047d0 0x7fd498004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.595 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b2089640 1 -- 172.21.15.122:0/384498906 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd498003530 con 0x7fd498003980 2024-03-16T20:45:58.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.587+0000 7fd4b288a640 1 --2- 172.21.15.122:0/384498906 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4ac066fa0 0x7fd498005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:58.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b2089640 1 --2- 172.21.15.122:0/384498906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd498003980 0x7fd498003da0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009b10 tx=0x7fd4a801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d33f970a9c0234 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:58.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b1087640 1 -- 172.21.15.122:0/384498906 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a80a6480 con 0x7fd498003980 2024-03-16T20:45:58.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b1087640 1 -- 172.21.15.122:0/384498906 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a80a65e0 con 0x7fd498003980 2024-03-16T20:45:58.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b1087640 1 -- 172.21.15.122:0/384498906 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a80a6990 con 0x7fd498003980 2024-03-16T20:45:58.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b42ed640 1 -- 172.21.15.122:0/384498906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd498003980 msgr2=0x7fd498003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:58.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b42ed640 1 --2- 172.21.15.122:0/384498906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd498003980 0x7fd498003da0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8009b10 tx=0x7fd4a801ca70 comp rx=0 tx=0).stop 2024-03-16T20:45:58.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b288a640 1 -- 172.21.15.122:0/384498906 reap_dead start 2024-03-16T20:45:58.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b42ed640 1 -- 172.21.15.122:0/384498906 shutdown_connections 2024-03-16T20:45:58.599 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b42ed640 1 -- 172.21.15.122:0/384498906 >> 172.21.15.122:0/384498906 conn(0x7fd4ac06d320 msgr2=0x7fd4ac06d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:58.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b42ed640 1 -- 172.21.15.122:0/384498906 shutdown_connections 2024-03-16T20:45:58.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b42ed640 1 -- 172.21.15.122:0/384498906 wait complete. 2024-03-16T20:45:58.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b42ed640 1 Processor -- start 2024-03-16T20:45:58.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b42ed640 1 -- start start 2024-03-16T20:45:58.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b42ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd498003980 0x7fd4ac1462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b42ed640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4ac140b80 0x7fd4ac14d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b42ed640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4ac14cfa0 0x7fd4ac13de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b42ed640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac14e730 con 0x7fd498003980 2024-03-16T20:45:58.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b42ed640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac14cb10 con 0x7fd4ac140b80 2024-03-16T20:45:58.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b42ed640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac146940 con 0x7fd4ac14cfa0 2024-03-16T20:45:58.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b2089640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd498003980 0x7fd4ac1462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b2089640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd498003980 0x7fd4ac1462e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40598/0 (socket says 172.21.15.122:40598) 2024-03-16T20:45:58.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b2089640 1 -- 172.21.15.122:0/3140760422 learned_addr learned my addr 172.21.15.122:0/3140760422 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:58.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b1888640 1 --2- 172.21.15.122:0/3140760422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4ac140b80 0x7fd4ac14d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b2089640 1 -- 172.21.15.122:0/3140760422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4ac14cfa0 msgr2=0x7fd4ac13de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b2089640 1 --2- 172.21.15.122:0/3140760422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4ac14cfa0 0x7fd4ac13de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b2089640 1 -- 172.21.15.122:0/3140760422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4ac140b80 msgr2=0x7fd4ac14d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b2089640 1 --2- 172.21.15.122:0/3140760422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4ac140b80 0x7fd4ac14d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b2089640 1 -- 172.21.15.122:0/3140760422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac13f750 con 0x7fd498003980 2024-03-16T20:45:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b1888640 1 --2- 172.21.15.122:0/3140760422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4ac140b80 0x7fd4ac14d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4b2089640 1 --2- 172.21.15.122:0/3140760422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd498003980 0x7fd4ac1462e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd4a801c8a0 tx=0x7fd498005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4a2ffd640 1 -- 172.21.15.122:0/3140760422 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a80a6d50 con 0x7fd498003980 2024-03-16T20:45:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4a2ffd640 1 -- 172.21.15.122:0/3140760422 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a80a7460 con 0x7fd498003980 2024-03-16T20:45:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.591+0000 7fd4a2ffd640 1 -- 172.21.15.122:0/3140760422 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a80a77f0 con 0x7fd498003980 2024-03-16T20:45:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.595+0000 7fd4b42ed640 1 -- 172.21.15.122:0/3140760422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ac146c20 con 0x7fd498003980 2024-03-16T20:45:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.595+0000 7fd4b42ed640 1 -- 172.21.15.122:0/3140760422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac14c790 con 0x7fd498003980 2024-03-16T20:45:58.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.595+0000 7fd4b42ed640 1 -- 172.21.15.122:0/3140760422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ac14e180 con 0x7fd498003980 2024-03-16T20:45:58.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.595+0000 7fd4a2ffd640 1 -- 172.21.15.122:0/3140760422 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fd4a80a7950 con 0x7fd498003980 2024-03-16T20:45:58.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.595+0000 7fd4a2ffd640 1 --2- 172.21.15.122:0/3140760422 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd490066e40 0x7fd490069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.595+0000 7fd4a2ffd640 1 -- 172.21.15.122:0/3140760422 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4a811e6c0 con 0x7fd498003980 2024-03-16T20:45:58.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.595+0000 7fd4b1888640 1 --2- 172.21.15.122:0/3140760422 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd490066e40 0x7fd490069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.595+0000 7fd4b1888640 1 --2- 172.21.15.122:0/3140760422 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd490066e40 0x7fd490069300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0011f0 tx=0x7fd49c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:58.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.599+0000 7fd4a2ffd640 1 -- 172.21.15.122:0/3140760422 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd4a80eb4a0 con 0x7fd498003980 2024-03-16T20:45:58.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.747+0000 7fd4b42ed640 1 -- 172.21.15.122:0/3140760422 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4ac075280 con 0x7fd498003980 2024-03-16T20:45:58.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.747+0000 7fd4a2ffd640 1 -- 172.21.15.122:0/3140760422 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+16705 (secure 0 0 0) 0x7fd4a80be090 con 0x7fd498003980 2024-03-16T20:45:58.754 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:45:58.754 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:45.942613+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:42.678541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:43.274469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:45:58.755 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.751+0000 7fd4b42ed640 1 -- 172.21.15.122:0/3140760422 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd490066e40 msgr2=0x7fd490069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:58.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.751+0000 7fd4b42ed640 1 --2- 172.21.15.122:0/3140760422 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd490066e40 0x7fd490069300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0011f0 tx=0x7fd49c009040 comp rx=0 tx=0).stop 2024-03-16T20:45:58.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.751+0000 7fd4b42ed640 1 -- 172.21.15.122:0/3140760422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd498003980 msgr2=0x7fd4ac1462e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:58.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.751+0000 7fd4b42ed640 1 --2- 172.21.15.122:0/3140760422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd498003980 0x7fd4ac1462e0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd4a801c8a0 tx=0x7fd498005800 comp rx=0 tx=0).stop 2024-03-16T20:45:58.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.751+0000 7fd4b42ed640 1 -- 172.21.15.122:0/3140760422 shutdown_connections 2024-03-16T20:45:58.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.751+0000 7fd4b42ed640 1 --2- 172.21.15.122:0/3140760422 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd490066e40 0x7fd490069300 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.751+0000 7fd4b42ed640 1 --2- 172.21.15.122:0/3140760422 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4ac14cfa0 0x7fd4ac13de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.751+0000 7fd4b42ed640 1 --2- 172.21.15.122:0/3140760422 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd4ac140b80 0x7fd4ac14d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.751+0000 7fd4b42ed640 1 --2- 172.21.15.122:0/3140760422 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd498003980 0x7fd4ac1462e0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.751+0000 7fd4b42ed640 1 -- 172.21.15.122:0/3140760422 >> 172.21.15.122:0/3140760422 conn(0x7fd4ac06d320 msgr2=0x7fd4ac0b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:58.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.751+0000 7fd4b42ed640 1 -- 172.21.15.122:0/3140760422 shutdown_connections 2024-03-16T20:45:58.756 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.751+0000 7fd4b42ed640 1 -- 172.21.15.122:0/3140760422 wait complete. 2024-03-16T20:45:58.778 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-16T20:44:52.653523+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-16T20:45:42.678541+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '29', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-03-16T20:45:43.274469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '30', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-16T20:45:58.779 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-16T20:45:58.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45fa989640 1 Processor -- start 2024-03-16T20:45:58.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45fa989640 1 -- start start 2024-03-16T20:45:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45fa989640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45f406f6f0 0x7f45f4141270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45fa989640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f45f41417b0 0x7f45f4151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45fa989640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45f4147ca0 con 0x7f45f406b1f0 2024-03-16T20:45:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45fa989640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45f40b7c60 con 0x7f45f41417b0 2024-03-16T20:45:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45fa989640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f406b5d0 con 0x7f45f406f6f0 2024-03-16T20:45:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45f8f26640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f45f41417b0 0x7f45f4151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45f8f26640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f45f41417b0 0x7f45f4151bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40612/0 (socket says 172.21.15.122:40612) 2024-03-16T20:45:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45f8f26640 1 -- 172.21.15.122:0/2663616044 learned_addr learned my addr 172.21.15.122:0/2663616044 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45f37fe640 1 --2- 172.21.15.122:0/2663616044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45f406f6f0 0x7f45f4141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45f8f26640 1 -- 172.21.15.122:0/2663616044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45f406f6f0 msgr2=0x7f45f4141270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45f8f26640 1 --2- 172.21.15.122:0/2663616044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45f406f6f0 0x7f45f4141270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45f8f26640 1 -- 172.21.15.122:0/2663616044 >> v1:172.21.15.195:6789/0 conn(0x7f45f406b1f0 legacy=0x7f45f40b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:58.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45f8f26640 1 -- 172.21.15.122:0/2663616044 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f4147650 con 0x7f45f41417b0 2024-03-16T20:45:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45f8f26640 1 --2- 172.21.15.122:0/2663616044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45f41417b0 0x7f45f4151bc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009e50 tx=0x7f45ec01ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe32870ed5e412b server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45f2ffd640 1 -- 172.21.15.122:0/2663616044 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0a5890 con 0x7f45f41417b0 2024-03-16T20:45:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45f2ffd640 1 -- 172.21.15.122:0/2663616044 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45ec0a59f0 con 0x7f45f41417b0 2024-03-16T20:45:58.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.891+0000 7f45f2ffd640 1 -- 172.21.15.122:0/2663616044 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45ec0a5d80 con 0x7f45f41417b0 2024-03-16T20:45:58.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 -- 172.21.15.122:0/2663616044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45f41417b0 msgr2=0x7f45f4151bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 --2- 172.21.15.122:0/2663616044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45f41417b0 0x7f45f4151bc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009e50 tx=0x7f45ec01ca00 comp rx=0 tx=0).stop 2024-03-16T20:45:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 -- 172.21.15.122:0/2663616044 shutdown_connections 2024-03-16T20:45:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 --2- 172.21.15.122:0/2663616044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45f41417b0 0x7f45f4151bc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 --2- 172.21.15.122:0/2663616044 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45f406f6f0 0x7f45f4141270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 -- 172.21.15.122:0/2663616044 >> 172.21.15.122:0/2663616044 conn(0x7f45f4070d30 msgr2=0x7f45f4071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 -- 172.21.15.122:0/2663616044 shutdown_connections 2024-03-16T20:45:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 -- 172.21.15.122:0/2663616044 wait complete. 2024-03-16T20:45:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 Processor -- start 2024-03-16T20:45:58.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 -- start start 2024-03-16T20:45:58.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45f406b1f0 0x7f45f4135510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f406f6f0 0x7f45f413cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45f41417b0 0x7f45f413d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f412f9a0 con 0x7f45f406b1f0 2024-03-16T20:45:58.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40b4d40 con 0x7f45f406f6f0 2024-03-16T20:45:58.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45f40b5d80 con 0x7f45f41417b0 2024-03-16T20:45:58.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45f3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45f406b1f0 0x7f45f4135510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45f3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45f406b1f0 0x7f45f4135510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40616/0 (socket says 172.21.15.122:40616) 2024-03-16T20:45:58.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45f3fff640 1 -- 172.21.15.122:0/1329870401 learned_addr learned my addr 172.21.15.122:0/1329870401 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:58.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45f37fe640 1 --2- 172.21.15.122:0/1329870401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f406f6f0 0x7f45f413cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45f8f26640 1 --2- 172.21.15.122:0/1329870401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45f41417b0 0x7f45f413d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45f3fff640 1 -- 172.21.15.122:0/1329870401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45f41417b0 msgr2=0x7f45f413d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:58.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45f3fff640 1 --2- 172.21.15.122:0/1329870401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45f41417b0 0x7f45f413d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45f3fff640 1 -- 172.21.15.122:0/1329870401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f406f6f0 msgr2=0x7f45f413cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:58.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45f3fff640 1 --2- 172.21.15.122:0/1329870401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f406f6f0 0x7f45f413cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:58.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45f3fff640 1 -- 172.21.15.122:0/1329870401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ec009b30 con 0x7f45f406b1f0 2024-03-16T20:45:58.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45f3fff640 1 --2- 172.21.15.122:0/1329870401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45f406b1f0 0x7f45f4135510 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f45e400c530 tx=0x7f45e400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45f0ff9640 1 -- 172.21.15.122:0/1329870401 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e4005220 con 0x7f45f406b1f0 2024-03-16T20:45:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45f0ff9640 1 -- 172.21.15.122:0/1329870401 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e4005380 con 0x7f45f406b1f0 2024-03-16T20:45:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 -- 172.21.15.122:0/1329870401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f40b4780 con 0x7f45f406b1f0 2024-03-16T20:45:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45f0ff9640 1 -- 172.21.15.122:0/1329870401 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e4007270 con 0x7f45f406b1f0 2024-03-16T20:45:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.895+0000 7f45fa989640 1 -- 172.21.15.122:0/1329870401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f40b5aa0 con 0x7f45f406b1f0 2024-03-16T20:45:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.899+0000 7f45fa989640 1 -- 172.21.15.122:0/1329870401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f40ba6b0 con 0x7f45f406b1f0 2024-03-16T20:45:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.899+0000 7f45f0ff9640 1 -- 172.21.15.122:0/1329870401 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f45e40073d0 con 0x7f45f406b1f0 2024-03-16T20:45:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.899+0000 7f45f0ff9640 1 --2- 172.21.15.122:0/1329870401 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f45e0066e40 0x7f45e0069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:58.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.899+0000 7f45f0ff9640 1 -- 172.21.15.122:0/1329870401 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45e4084360 con 0x7f45f406b1f0 2024-03-16T20:45:58.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.899+0000 7f45f37fe640 1 --2- 172.21.15.122:0/1329870401 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f45e0066e40 0x7f45e0069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:58.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.899+0000 7f45f37fe640 1 --2- 172.21.15.122:0/1329870401 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f45e0066e40 0x7f45e0069300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f45f4145240 tx=0x7f45e8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:58.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:58.903+0000 7f45f0ff9640 1 -- 172.21.15.122:0/1329870401 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f45e4051180 con 0x7f45f406b1f0 2024-03-16T20:45:59.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.063+0000 7f45fa989640 1 -- 172.21.15.122:0/1329870401 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f45f4074bb0 con 0x7f45f406b1f0 2024-03-16T20:45:59.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.063+0000 7f45f0ff9640 1 -- 172.21.15.122:0/1329870401 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f45e4054840 con 0x7f45f406b1f0 2024-03-16T20:45:59.068 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 1 2024-03-16T20:45:59.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.063+0000 7f45fa989640 1 -- 172.21.15.122:0/1329870401 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f45e0066e40 msgr2=0x7f45e0069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.063+0000 7f45fa989640 1 --2- 172.21.15.122:0/1329870401 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f45e0066e40 0x7f45e0069300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f45f4145240 tx=0x7f45e8008040 comp rx=0 tx=0).stop 2024-03-16T20:45:59.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.063+0000 7f45fa989640 1 -- 172.21.15.122:0/1329870401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45f406b1f0 msgr2=0x7f45f4135510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.063+0000 7f45fa989640 1 --2- 172.21.15.122:0/1329870401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45f406b1f0 0x7f45f4135510 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f45e400c530 tx=0x7f45e400c9f0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.063+0000 7f45fa989640 1 -- 172.21.15.122:0/1329870401 shutdown_connections 2024-03-16T20:45:59.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.063+0000 7f45fa989640 1 --2- 172.21.15.122:0/1329870401 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f45e0066e40 0x7f45e0069300 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.063+0000 7f45fa989640 1 --2- 172.21.15.122:0/1329870401 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f45f41417b0 0x7f45f413d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.063+0000 7f45fa989640 1 --2- 172.21.15.122:0/1329870401 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f45f406f6f0 0x7f45f413cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.063+0000 7f45fa989640 1 --2- 172.21.15.122:0/1329870401 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45f406b1f0 0x7f45f4135510 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.063+0000 7f45fa989640 1 -- 172.21.15.122:0/1329870401 >> 172.21.15.122:0/1329870401 conn(0x7f45f4070d30 msgr2=0x7f45f40b8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:59.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.067+0000 7f45fa989640 1 -- 172.21.15.122:0/1329870401 shutdown_connections 2024-03-16T20:45:59.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.067+0000 7f45fa989640 1 -- 172.21.15.122:0/1329870401 wait complete. 2024-03-16T20:45:59.086 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-16T20:45:59.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.195+0000 7ff2213b9640 1 Processor -- start 2024-03-16T20:45:59.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.195+0000 7ff2213b9640 1 -- start start 2024-03-16T20:45:59.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.195+0000 7ff2213b9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7ff21c06d350 0x7ff21c152930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.195+0000 7ff2213b9640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff21c147ff0 con 0x7ff21c141580 2024-03-16T20:45:59.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.195+0000 7ff2213b9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff21c1479a0 con 0x7ff21c0bb1d0 2024-03-16T20:45:59.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.195+0000 7ff2213b9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff21c147c80 con 0x7ff21c06d350 2024-03-16T20:45:59.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.195+0000 7ff21affd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7ff21c141580 0x7ff21c0baac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:40498/0 (socket says 172.21.15.122:40498) 2024-03-16T20:45:59.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.195+0000 7ff21affd640 1 -- 172.21.15.122:0/573465144 learned_addr learned my addr 172.21.15.122:0/573465144 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:59.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 -- 172.21.15.122:0/573465144 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3400711375 0 0) 0x7ff21c147ff0 con 0x7ff21c141580 2024-03-16T20:45:59.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21b7fe640 1 --2- 172.21.15.122:0/573465144 >> v2:172.21.15.195:3300/0 conn(0x7ff21c06d350 0x7ff21c152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.204 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 -- 172.21.15.122:0/573465144 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff208003530 con 0x7ff21c141580 2024-03-16T20:45:59.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 -- 172.21.15.122:0/573465144 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1791651141 0 0) 0x7ff208003530 con 0x7ff21c141580 2024-03-16T20:45:59.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 -- 172.21.15.122:0/573465144 >> v2:172.21.15.195:3300/0 conn(0x7ff21c06d350 msgr2=0x7ff21c152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:59.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 --2- 172.21.15.122:0/573465144 >> v2:172.21.15.195:3300/0 conn(0x7ff21c06d350 0x7ff21c152930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 -- 172.21.15.122:0/573465144 >> v1:172.21.15.195:6789/0 conn(0x7ff21c0bb1d0 legacy=0x7ff21c06cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 -- 172.21.15.122:0/573465144 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c06d7e0 con 0x7ff21c141580 2024-03-16T20:45:59.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 -- 172.21.15.122:0/573465144 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7ff20c001680 con 0x7ff21c141580 2024-03-16T20:45:59.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 -- 172.21.15.122:0/573465144 >> v1:172.21.15.122:6789/0 conn(0x7ff21c141580 legacy=0x7ff21c0baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 --2- 172.21.15.122:0/573465144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff208003980 0x7ff208003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 --2- 172.21.15.122:0/573465144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2080047d0 0x7ff208004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.205 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 --2- 172.21.15.122:0/573465144 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff208005190 0x7ff20800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 -- 172.21.15.122:0/573465144 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c147c80 con 0x7ff2080047d0 2024-03-16T20:45:59.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 -- 172.21.15.122:0/573465144 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff20800c030 con 0x7ff208003980 2024-03-16T20:45:59.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 -- 172.21.15.122:0/573465144 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20800c1f0 con 0x7ff208005190 2024-03-16T20:45:59.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff2127fc640 1 --2- 172.21.15.122:0/573465144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2080047d0 0x7ff208004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff2127fc640 1 -- 172.21.15.122:0/573465144 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff208005190 msgr2=0x7ff20800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:59.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff2127fc640 1 --2- 172.21.15.122:0/573465144 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff208005190 0x7ff20800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff2127fc640 1 -- 172.21.15.122:0/573465144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff208003980 msgr2=0x7ff208003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:59.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff2127fc640 1 --2- 172.21.15.122:0/573465144 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff208003980 0x7ff208003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff2127fc640 1 -- 172.21.15.122:0/573465144 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff208003530 con 0x7ff2080047d0 2024-03-16T20:45:59.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21b7fe640 1 -- 172.21.15.122:0/573465144 reap_dead start 2024-03-16T20:45:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff2127fc640 1 --2- 172.21.15.122:0/573465144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2080047d0 0x7ff208004c50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff204001750 tx=0x7ff20401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5f7752cdaac568 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 -- 172.21.15.122:0/573465144 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2040052d0 con 0x7ff2080047d0 2024-03-16T20:45:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 -- 172.21.15.122:0/573465144 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2040a7050 con 0x7ff2080047d0 2024-03-16T20:45:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff21a7fc640 1 -- 172.21.15.122:0/573465144 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff204003600 con 0x7ff2080047d0 2024-03-16T20:45:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff2213b9640 1 -- 172.21.15.122:0/573465144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2080047d0 msgr2=0x7ff208004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff2213b9640 1 --2- 172.21.15.122:0/573465144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2080047d0 0x7ff208004c50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff204001750 tx=0x7ff20401c3c0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff2213b9640 1 -- 172.21.15.122:0/573465144 shutdown_connections 2024-03-16T20:45:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff2213b9640 1 --2- 172.21.15.122:0/573465144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff2080047d0 0x7ff208004c50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff2213b9640 1 -- 172.21.15.122:0/573465144 >> 172.21.15.122:0/573465144 conn(0x7ff21c0bca20 msgr2=0x7ff21c0bce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:59.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff2213b9640 1 -- 172.21.15.122:0/573465144 shutdown_connections 2024-03-16T20:45:59.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.199+0000 7ff2213b9640 1 -- 172.21.15.122:0/573465144 wait complete. 2024-03-16T20:45:59.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2213b9640 1 Processor -- start 2024-03-16T20:45:59.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2213b9640 1 -- start start 2024-03-16T20:45:59.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2213b9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2080047d0 0x7ff21c0b4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2213b9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff21c141b20 0x7ff21c1dc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2213b9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff21c1446e0 0x7ff21c1dce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2213b9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c0b5f50 con 0x7ff21c141b20 2024-03-16T20:45:59.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2213b9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c0b6370 con 0x7ff21c1446e0 2024-03-16T20:45:59.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2213b9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c142000 con 0x7ff2080047d0 2024-03-16T20:45:59.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2127fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff21c141b20 0x7ff21c1dc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2127fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff21c141b20 0x7ff21c1dc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40636/0 (socket says 172.21.15.122:40636) 2024-03-16T20:45:59.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2127fc640 1 -- 172.21.15.122:0/2837398463 learned_addr learned my addr 172.21.15.122:0/2837398463 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:59.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff21affd640 1 --2- 172.21.15.122:0/2837398463 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2080047d0 0x7ff21c0b4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff21b7fe640 1 --2- 172.21.15.122:0/2837398463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff21c1446e0 0x7ff21c1dce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2127fc640 1 -- 172.21.15.122:0/2837398463 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2080047d0 msgr2=0x7ff21c0b4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2127fc640 1 --2- 172.21.15.122:0/2837398463 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2080047d0 0x7ff21c0b4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2127fc640 1 -- 172.21.15.122:0/2837398463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff21c1446e0 msgr2=0x7ff21c1dce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2127fc640 1 --2- 172.21.15.122:0/2837398463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff21c1446e0 0x7ff21c1dce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2127fc640 1 -- 172.21.15.122:0/2837398463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c0b5890 con 0x7ff21c141b20 2024-03-16T20:45:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff21b7fe640 1 --2- 172.21.15.122:0/2837398463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff21c1446e0 0x7ff21c1dce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff21affd640 1 --2- 172.21.15.122:0/2837398463 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2080047d0 0x7ff21c0b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:45:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2127fc640 1 --2- 172.21.15.122:0/2837398463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff21c141b20 0x7ff21c1dc920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff204005040 tx=0x7ff208005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff213fff640 1 -- 172.21.15.122:0/2837398463 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff204007500 con 0x7ff21c141b20 2024-03-16T20:45:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff213fff640 1 -- 172.21.15.122:0/2837398463 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff204007660 con 0x7ff21c141b20 2024-03-16T20:45:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff213fff640 1 -- 172.21.15.122:0/2837398463 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff204006a00 con 0x7ff21c141b20 2024-03-16T20:45:59.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2213b9640 1 -- 172.21.15.122:0/2837398463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21c0b5b70 con 0x7ff21c141b20 2024-03-16T20:45:59.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.203+0000 7ff2213b9640 1 -- 172.21.15.122:0/2837398463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21c143810 con 0x7ff21c141b20 2024-03-16T20:45:59.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.207+0000 7ff2213b9640 1 -- 172.21.15.122:0/2837398463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21c0684d0 con 0x7ff21c141b20 2024-03-16T20:45:59.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.207+0000 7ff213fff640 1 -- 172.21.15.122:0/2837398463 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7ff204006b60 con 0x7ff21c141b20 2024-03-16T20:45:59.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.207+0000 7ff213fff640 1 --2- 172.21.15.122:0/2837398463 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff1fc066e40 0x7ff1fc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.207+0000 7ff213fff640 1 -- 172.21.15.122:0/2837398463 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff20411ec90 con 0x7ff21c141b20 2024-03-16T20:45:59.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.207+0000 7ff21affd640 1 --2- 172.21.15.122:0/2837398463 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff1fc066e40 0x7ff1fc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.207+0000 7ff21affd640 1 --2- 172.21.15.122:0/2837398463 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff1fc066e40 0x7ff1fc069300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0017b0 tx=0x7ff20c0015a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:59.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.211+0000 7ff213fff640 1 -- 172.21.15.122:0/2837398463 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7ff2040ebb10 con 0x7ff21c141b20 2024-03-16T20:45:59.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.507+0000 7ff2213b9640 1 -- 172.21.15.122:0/2837398463 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff21c074bb0 con 0x7ff21c141b20 2024-03-16T20:45:59.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.511+0000 7ff213fff640 1 -- 172.21.15.122:0/2837398463 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v30) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff2040ef1d0 con 0x7ff21c141b20 2024-03-16T20:45:59.515 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 32 2024-03-16T20:45:59.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.511+0000 7ff2213b9640 1 -- 172.21.15.122:0/2837398463 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff1fc066e40 msgr2=0x7ff1fc069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.511+0000 7ff2213b9640 1 --2- 172.21.15.122:0/2837398463 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff1fc066e40 0x7ff1fc069300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff20c0017b0 tx=0x7ff20c0015a0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.511+0000 7ff2213b9640 1 -- 172.21.15.122:0/2837398463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff21c141b20 msgr2=0x7ff21c1dc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.511+0000 7ff2213b9640 1 --2- 172.21.15.122:0/2837398463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff21c141b20 0x7ff21c1dc920 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff204005040 tx=0x7ff208005800 comp rx=0 tx=0).stop 2024-03-16T20:45:59.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.511+0000 7ff2213b9640 1 -- 172.21.15.122:0/2837398463 shutdown_connections 2024-03-16T20:45:59.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.511+0000 7ff2213b9640 1 --2- 172.21.15.122:0/2837398463 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7ff1fc066e40 0x7ff1fc069300 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.511+0000 7ff2213b9640 1 --2- 172.21.15.122:0/2837398463 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7ff21c1446e0 0x7ff21c1dce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.511+0000 7ff2213b9640 1 --2- 172.21.15.122:0/2837398463 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff21c141b20 0x7ff21c1dc920 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.511+0000 7ff2213b9640 1 --2- 172.21.15.122:0/2837398463 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7ff2080047d0 0x7ff21c0b4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.511+0000 7ff2213b9640 1 -- 172.21.15.122:0/2837398463 >> 172.21.15.122:0/2837398463 conn(0x7ff21c0bca20 msgr2=0x7ff21c0b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:59.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.511+0000 7ff2213b9640 1 -- 172.21.15.122:0/2837398463 shutdown_connections 2024-03-16T20:45:59.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.511+0000 7ff2213b9640 1 -- 172.21.15.122:0/2837398463 wait complete. 2024-03-16T20:45:59.539 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-16T20:45:59.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e885d1640 1 Processor -- start 2024-03-16T20:45:59.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e885d1640 1 -- start start 2024-03-16T20:45:59.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e885d1640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f3e800adf80 0x7f3e80072510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e885d1640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e800aec70 con 0x7f3e80066fa0 2024-03-16T20:45:59.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e885d1640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e800ae690 con 0x7f3e8014b1b0 2024-03-16T20:45:59.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e885d1640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e80067380 con 0x7f3e800adf80 2024-03-16T20:45:59.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8636d640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f3e80066fa0 0x7f3e8014aaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:40502/0 (socket says 172.21.15.122:40502) 2024-03-16T20:45:59.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8636d640 1 -- 172.21.15.122:0/1637391740 learned_addr learned my addr 172.21.15.122:0/1637391740 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:45:59.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8536b640 1 -- 172.21.15.122:0/1637391740 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76057638 0 0) 0x7f3e800aec70 con 0x7f3e80066fa0 2024-03-16T20:45:59.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8536b640 1 -- 172.21.15.122:0/1637391740 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e6c003530 con 0x7f3e80066fa0 2024-03-16T20:45:59.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e85b6c640 1 --2- 172.21.15.122:0/1637391740 >> v2:172.21.15.195:3300/0 conn(0x7f3e800adf80 0x7f3e80072510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8536b640 1 -- 172.21.15.122:0/1637391740 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2555704457 0 0) 0x7f3e6c003530 con 0x7f3e80066fa0 2024-03-16T20:45:59.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8536b640 1 -- 172.21.15.122:0/1637391740 >> v2:172.21.15.195:3300/0 conn(0x7f3e800adf80 msgr2=0x7f3e80072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:59.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8536b640 1 --2- 172.21.15.122:0/1637391740 >> v2:172.21.15.195:3300/0 conn(0x7f3e800adf80 0x7f3e80072510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8536b640 1 -- 172.21.15.122:0/1637391740 >> v1:172.21.15.195:6789/0 conn(0x7f3e8014b1b0 legacy=0x7f3e80151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8536b640 1 -- 172.21.15.122:0/1637391740 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e800afa90 con 0x7f3e80066fa0 2024-03-16T20:45:59.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8536b640 1 -- 172.21.15.122:0/1637391740 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f3e7c001a00 con 0x7f3e80066fa0 2024-03-16T20:45:59.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8536b640 1 -- 172.21.15.122:0/1637391740 >> v1:172.21.15.122:6789/0 conn(0x7f3e80066fa0 legacy=0x7f3e8014aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8536b640 1 --2- 172.21.15.122:0/1637391740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e6c003980 0x7f3e6c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8536b640 1 --2- 172.21.15.122:0/1637391740 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e6c0047d0 0x7f3e6c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8536b640 1 --2- 172.21.15.122:0/1637391740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e6c005190 0x7f3e6c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8536b640 1 -- 172.21.15.122:0/1637391740 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e80067380 con 0x7f3e6c005190 2024-03-16T20:45:59.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8536b640 1 -- 172.21.15.122:0/1637391740 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c00c130 con 0x7f3e6c003980 2024-03-16T20:45:59.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8536b640 1 -- 172.21.15.122:0/1637391740 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c00c2f0 con 0x7f3e6c0047d0 2024-03-16T20:45:59.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e86b6e640 1 --2- 172.21.15.122:0/1637391740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e6c005190 0x7f3e6c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e8636d640 1 --2- 172.21.15.122:0/1637391740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e6c003980 0x7f3e6c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e86b6e640 1 -- 172.21.15.122:0/1637391740 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e6c0047d0 msgr2=0x7f3e6c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:59.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e86b6e640 1 --2- 172.21.15.122:0/1637391740 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e6c0047d0 0x7f3e6c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.655 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e86b6e640 1 -- 172.21.15.122:0/1637391740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e6c003980 msgr2=0x7f3e6c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:59.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e86b6e640 1 --2- 172.21.15.122:0/1637391740 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e6c003980 0x7f3e6c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.656 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.647+0000 7f3e86b6e640 1 -- 172.21.15.122:0/1637391740 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c003530 con 0x7f3e6c005190 2024-03-16T20:45:59.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e86b6e640 1 -- 172.21.15.122:0/1637391740 reap_dead start 2024-03-16T20:45:59.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e86b6e640 1 --2- 172.21.15.122:0/1637391740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e6c005190 0x7f3e6c00b990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3e78001780 tx=0x7f3e7801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ad7df06b98870ce server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:59.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e8536b640 1 -- 172.21.15.122:0/1637391740 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e780052d0 con 0x7f3e6c005190 2024-03-16T20:45:59.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e8536b640 1 -- 172.21.15.122:0/1637391740 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e780a7050 con 0x7f3e6c005190 2024-03-16T20:45:59.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e8536b640 1 -- 172.21.15.122:0/1637391740 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e78003600 con 0x7f3e6c005190 2024-03-16T20:45:59.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e885d1640 1 -- 172.21.15.122:0/1637391740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e6c005190 msgr2=0x7f3e6c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e885d1640 1 --2- 172.21.15.122:0/1637391740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e6c005190 0x7f3e6c00b990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3e78001780 tx=0x7f3e7801c420 comp rx=0 tx=0).stop 2024-03-16T20:45:59.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e885d1640 1 -- 172.21.15.122:0/1637391740 shutdown_connections 2024-03-16T20:45:59.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e885d1640 1 --2- 172.21.15.122:0/1637391740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e6c005190 0x7f3e6c00b990 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e885d1640 1 -- 172.21.15.122:0/1637391740 >> 172.21.15.122:0/1637391740 conn(0x7f3e8006d2b0 msgr2=0x7f3e8006d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:59.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e885d1640 1 -- 172.21.15.122:0/1637391740 shutdown_connections 2024-03-16T20:45:59.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e885d1640 1 -- 172.21.15.122:0/1637391740 wait complete. 2024-03-16T20:45:59.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e885d1640 1 Processor -- start 2024-03-16T20:45:59.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e885d1640 1 -- start start 2024-03-16T20:45:59.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e885d1640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e6c005190 0x7f3e800716a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e885d1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e800b1fd0 0x7f3e801df2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e885d1640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e801447c0 0x7f3e80144c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e885d1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8009cb70 con 0x7f3e800b1fd0 2024-03-16T20:45:59.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e885d1640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e800b2460 con 0x7f3e801447c0 2024-03-16T20:45:59.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e885d1640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e800b2700 con 0x7f3e6c005190 2024-03-16T20:45:59.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e85b6c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e800b1fd0 0x7f3e801df2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e85b6c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e800b1fd0 0x7f3e801df2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40656/0 (socket says 172.21.15.122:40656) 2024-03-16T20:45:59.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e85b6c640 1 -- 172.21.15.122:0/1468259410 learned_addr learned my addr 172.21.15.122:0/1468259410 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:59.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e8636d640 1 --2- 172.21.15.122:0/1468259410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e6c005190 0x7f3e800716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e86b6e640 1 --2- 172.21.15.122:0/1468259410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e801447c0 0x7f3e80144c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e85b6c640 1 -- 172.21.15.122:0/1468259410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e6c005190 msgr2=0x7f3e800716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e85b6c640 1 --2- 172.21.15.122:0/1468259410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e6c005190 0x7f3e800716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e85b6c640 1 -- 172.21.15.122:0/1468259410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e801447c0 msgr2=0x7f3e80144c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e85b6c640 1 --2- 172.21.15.122:0/1468259410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e801447c0 0x7f3e80144c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e85b6c640 1 -- 172.21.15.122:0/1468259410 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e800b1ab0 con 0x7f3e800b1fd0 2024-03-16T20:45:59.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e85b6c640 1 --2- 172.21.15.122:0/1468259410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e800b1fd0 0x7f3e801df2a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000e7c0 tx=0x7f3e7000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:59.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e76ffd640 1 -- 172.21.15.122:0/1468259410 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7000ee70 con 0x7f3e800b1fd0 2024-03-16T20:45:59.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e76ffd640 1 -- 172.21.15.122:0/1468259410 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7000c940 con 0x7f3e800b1fd0 2024-03-16T20:45:59.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e885d1640 1 -- 172.21.15.122:0/1468259410 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e801df7e0 con 0x7f3e800b1fd0 2024-03-16T20:45:59.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e76ffd640 1 -- 172.21.15.122:0/1468259410 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7000ccf0 con 0x7f3e800b1fd0 2024-03-16T20:45:59.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.651+0000 7f3e885d1640 1 -- 172.21.15.122:0/1468259410 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e800724a0 con 0x7f3e800b1fd0 2024-03-16T20:45:59.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.655+0000 7f3e885d1640 1 -- 172.21.15.122:0/1468259410 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e800b1910 con 0x7f3e800b1fd0 2024-03-16T20:45:59.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.655+0000 7f3e76ffd640 1 -- 172.21.15.122:0/1468259410 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f3e70013470 con 0x7f3e800b1fd0 2024-03-16T20:45:59.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.655+0000 7f3e76ffd640 1 --2- 172.21.15.122:0/1468259410 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3e64066e70 0x7f3e64069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.663 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.655+0000 7f3e76ffd640 1 -- 172.21.15.122:0/1468259410 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e70084a40 con 0x7f3e800b1fd0 2024-03-16T20:45:59.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.655+0000 7f3e8636d640 1 --2- 172.21.15.122:0/1468259410 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3e64066e70 0x7f3e64069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.655+0000 7f3e8636d640 1 --2- 172.21.15.122:0/1468259410 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3e64066e70 0x7f3e64069330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c001340 tx=0x7f3e7c001cf0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:59.664 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.659+0000 7f3e76ffd640 1 -- 172.21.15.122:0/1468259410 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3e800b1910 con 0x7f3e800b1fd0 2024-03-16T20:45:59.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.803+0000 7f3e885d1640 1 -- 172.21.15.122:0/1468259410 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3e800b1910 con 0x7f3e800b1fd0 2024-03-16T20:45:59.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.803+0000 7f3e76ffd640 1 -- 172.21.15.122:0/1468259410 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v30) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3e800b1910 con 0x7f3e800b1fd0 2024-03-16T20:45:59.809 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 64 2024-03-16T20:45:59.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.807+0000 7f3e885d1640 1 -- 172.21.15.122:0/1468259410 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3e64066e70 msgr2=0x7f3e64069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.807+0000 7f3e885d1640 1 --2- 172.21.15.122:0/1468259410 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3e64066e70 0x7f3e64069330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c001340 tx=0x7f3e7c001cf0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.807+0000 7f3e885d1640 1 -- 172.21.15.122:0/1468259410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e800b1fd0 msgr2=0x7f3e801df2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.807+0000 7f3e885d1640 1 --2- 172.21.15.122:0/1468259410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e800b1fd0 0x7f3e801df2a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000e7c0 tx=0x7f3e7000ec80 comp rx=0 tx=0).stop 2024-03-16T20:45:59.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.807+0000 7f3e885d1640 1 -- 172.21.15.122:0/1468259410 shutdown_connections 2024-03-16T20:45:59.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.807+0000 7f3e885d1640 1 --2- 172.21.15.122:0/1468259410 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3e64066e70 0x7f3e64069330 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.807+0000 7f3e885d1640 1 --2- 172.21.15.122:0/1468259410 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3e801447c0 0x7f3e80144c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.807+0000 7f3e885d1640 1 --2- 172.21.15.122:0/1468259410 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e800b1fd0 0x7f3e801df2a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.807+0000 7f3e885d1640 1 --2- 172.21.15.122:0/1468259410 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3e6c005190 0x7f3e800716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.807+0000 7f3e885d1640 1 -- 172.21.15.122:0/1468259410 >> 172.21.15.122:0/1468259410 conn(0x7f3e8006d2b0 msgr2=0x7f3e800b6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:59.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.807+0000 7f3e885d1640 1 -- 172.21.15.122:0/1468259410 shutdown_connections 2024-03-16T20:45:59.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.807+0000 7f3e885d1640 1 -- 172.21.15.122:0/1468259410 wait complete. 2024-03-16T20:45:59.827 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T20:45:59.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.927+0000 7fa5d8a8f640 1 Processor -- start 2024-03-16T20:45:59.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.927+0000 7fa5d8a8f640 1 -- start start 2024-03-16T20:45:59.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.927+0000 7fa5d8a8f640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa5d40baca0 0x7fa5d40b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.927+0000 7fa5d8a8f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5d4140c70 0x7fa5d4141100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.927+0000 7fa5d8a8f640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d40b1390 con 0x7fa5d4147080 2024-03-16T20:45:59.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.927+0000 7fa5d8a8f640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d40bb080 con 0x7fa5d40baca0 2024-03-16T20:45:59.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.927+0000 7fa5d8a8f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d406d9c0 con 0x7fa5d4140c70 2024-03-16T20:45:59.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.931+0000 7fa5d2d9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5d4140c70 0x7fa5d4141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.934 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.931+0000 7fa5d259c640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fa5d40baca0 0x7fa5d40b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.931+0000 7fa5d2d9d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5d4140c70 0x7fa5d4141100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:48190/0 (socket says 172.21.15.122:48190) 2024-03-16T20:45:59.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.931+0000 7fa5d2d9d640 1 -- 172.21.15.122:0/2593749030 learned_addr learned my addr 172.21.15.122:0/2593749030 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:59.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.931+0000 7fa5d2d9d640 1 -- 172.21.15.122:0/2593749030 >> v2:172.21.15.195:3300/0 conn(0x7fa5d40baca0 msgr2=0x7fa5d40b1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:45:59.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.931+0000 7fa5d2d9d640 1 --2- 172.21.15.122:0/2593749030 >> v2:172.21.15.195:3300/0 conn(0x7fa5d40baca0 0x7fa5d40b1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.931+0000 7fa5d2d9d640 1 -- 172.21.15.122:0/2593749030 >> v1:172.21.15.195:6789/0 conn(0x7fa5d4147080 legacy=0x7fa5d41474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.931+0000 7fa5d2d9d640 1 -- 172.21.15.122:0/2593749030 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d406dca0 con 0x7fa5d4140c70 2024-03-16T20:45:59.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.931+0000 7fa5d2d9d640 1 --2- 172.21.15.122:0/2593749030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5d4140c70 0x7fa5d4141100 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009e50 tx=0x7fa5cc01ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fade5d42ce0411d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:59.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.931+0000 7fa5d1d9b640 1 -- 172.21.15.122:0/2593749030 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc0a5810 con 0x7fa5d4140c70 2024-03-16T20:45:59.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.931+0000 7fa5d1d9b640 1 -- 172.21.15.122:0/2593749030 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc0a5970 con 0x7fa5d4140c70 2024-03-16T20:45:59.937 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.931+0000 7fa5d1d9b640 1 -- 172.21.15.122:0/2593749030 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc0a5d00 con 0x7fa5d4140c70 2024-03-16T20:45:59.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.931+0000 7fa5d8a8f640 1 -- 172.21.15.122:0/2593749030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5d4140c70 msgr2=0x7fa5d4141100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.931+0000 7fa5d8a8f640 1 --2- 172.21.15.122:0/2593749030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5d4140c70 0x7fa5d4141100 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009e50 tx=0x7fa5cc01ca30 comp rx=0 tx=0).stop 2024-03-16T20:45:59.938 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d8a8f640 1 -- 172.21.15.122:0/2593749030 shutdown_connections 2024-03-16T20:45:59.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d8a8f640 1 --2- 172.21.15.122:0/2593749030 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5d4140c70 0x7fa5d4141100 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d8a8f640 1 --2- 172.21.15.122:0/2593749030 >> v2:172.21.15.195:3300/0 conn(0x7fa5d40baca0 0x7fa5d40b1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d8a8f640 1 -- 172.21.15.122:0/2593749030 >> 172.21.15.122:0/2593749030 conn(0x7fa5d406ee40 msgr2=0x7fa5d406f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:45:59.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d8a8f640 1 -- 172.21.15.122:0/2593749030 shutdown_connections 2024-03-16T20:45:59.939 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d8a8f640 1 -- 172.21.15.122:0/2593749030 wait complete. 2024-03-16T20:45:59.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d8a8f640 1 Processor -- start 2024-03-16T20:45:59.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d8a8f640 1 -- start start 2024-03-16T20:45:59.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d8a8f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5d40baca0 0x7fa5d41df720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d8a8f640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5d4140c70 0x7fa5d413f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.941 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d8a8f640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5d4147080 0x7fa5d414a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d8a8f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40b3c10 con 0x7fa5d40baca0 2024-03-16T20:45:59.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d8a8f640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40b4140 con 0x7fa5d4140c70 2024-03-16T20:45:59.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d8a8f640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40b2180 con 0x7fa5d4147080 2024-03-16T20:45:59.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5d40baca0 0x7fa5d41df720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d259c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5d40baca0 0x7fa5d41df720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40658/0 (socket says 172.21.15.122:40658) 2024-03-16T20:45:59.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d259c640 1 -- 172.21.15.122:0/2613418135 learned_addr learned my addr 172.21.15.122:0/2613418135 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:45:59.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d2d9d640 1 --2- 172.21.15.122:0/2613418135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5d4147080 0x7fa5d414a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5cbfff640 1 --2- 172.21.15.122:0/2613418135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5d4140c70 0x7fa5d413f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d259c640 1 -- 172.21.15.122:0/2613418135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5d4147080 msgr2=0x7fa5d414a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d259c640 1 --2- 172.21.15.122:0/2613418135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5d4147080 0x7fa5d414a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d259c640 1 -- 172.21.15.122:0/2613418135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5d4140c70 msgr2=0x7fa5d413f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:45:59.942 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d259c640 1 --2- 172.21.15.122:0/2613418135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5d4140c70 0x7fa5d413f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:45:59.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d259c640 1 -- 172.21.15.122:0/2613418135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc009b30 con 0x7fa5d40baca0 2024-03-16T20:45:59.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5cbfff640 1 --2- 172.21.15.122:0/2613418135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5d4140c70 0x7fa5d413f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:45:59.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d259c640 1 --2- 172.21.15.122:0/2613418135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5d40baca0 0x7fa5d41df720 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400e930 tx=0x7fa5c400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:59.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5caffd640 1 -- 172.21.15.122:0/2613418135 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4009690 con 0x7fa5d40baca0 2024-03-16T20:45:59.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5caffd640 1 -- 172.21.15.122:0/2613418135 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c400ca00 con 0x7fa5d40baca0 2024-03-16T20:45:59.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d8a8f640 1 -- 172.21.15.122:0/2613418135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d40b2460 con 0x7fa5d40baca0 2024-03-16T20:45:59.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5caffd640 1 -- 172.21.15.122:0/2613418135 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c400cd70 con 0x7fa5d40baca0 2024-03-16T20:45:59.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d8a8f640 1 -- 172.21.15.122:0/2613418135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d413ea90 con 0x7fa5d40baca0 2024-03-16T20:45:59.943 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5d8a8f640 1 -- 172.21.15.122:0/2613418135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d40677b0 con 0x7fa5d40baca0 2024-03-16T20:45:59.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.935+0000 7fa5caffd640 1 -- 172.21.15.122:0/2613418135 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fa5c4013470 con 0x7fa5d40baca0 2024-03-16T20:45:59.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.939+0000 7fa5caffd640 1 --2- 172.21.15.122:0/2613418135 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa5b4066e70 0x7fa5b4069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:45:59.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.939+0000 7fa5caffd640 1 -- 172.21.15.122:0/2613418135 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5c4084bb0 con 0x7fa5d40baca0 2024-03-16T20:45:59.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.939+0000 7fa5cbfff640 1 --2- 172.21.15.122:0/2613418135 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa5b4066e70 0x7fa5b4069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:45:59.944 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.939+0000 7fa5cbfff640 1 --2- 172.21.15.122:0/2613418135 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa5b4066e70 0x7fa5b4069330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4144fc0 tx=0x7fa5bc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:45:59.946 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:45:59.943+0000 7fa5caffd640 1 -- 172.21.15.122:0/2613418135 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fa5c4051af0 con 0x7fa5d40baca0 2024-03-16T20:46:00.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.123+0000 7fa5d8a8f640 1 -- 172.21.15.122:0/2613418135 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5d41dfcc0 con 0x7fa5d40baca0 2024-03-16T20:46:00.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.123+0000 7fa5caffd640 1 -- 172.21.15.122:0/2613418135 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+3401 (secure 0 0 0) 0x7fa5c4021d60 con 0x7fa5d40baca0 2024-03-16T20:46:00.129 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:46:00.129 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3638578490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3638578490},{"type":"v1","addr":"172.21.15.195:6835","nonce":3638578490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T20:45:44.280433+0000","modified":"2024-03-16T20:45:46.325382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4995":{"gid":4995,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6833/1841055069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1841055069},{"type":"v1","addr":"172.21.15.122:6833","nonce":1841055069}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4995,"qdb_cluster":[4995]},"id":2}]} 2024-03-16T20:46:00.129 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 13 2024-03-16T20:46:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.127+0000 7fa5d8a8f640 1 -- 172.21.15.122:0/2613418135 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa5b4066e70 msgr2=0x7fa5b4069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.127+0000 7fa5d8a8f640 1 --2- 172.21.15.122:0/2613418135 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa5b4066e70 0x7fa5b4069330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa5d4144fc0 tx=0x7fa5bc009040 comp rx=0 tx=0).stop 2024-03-16T20:46:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.127+0000 7fa5d8a8f640 1 -- 172.21.15.122:0/2613418135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5d40baca0 msgr2=0x7fa5d41df720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.127+0000 7fa5d8a8f640 1 --2- 172.21.15.122:0/2613418135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5d40baca0 0x7fa5d41df720 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400e930 tx=0x7fa5c400edf0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.127+0000 7fa5d8a8f640 1 -- 172.21.15.122:0/2613418135 shutdown_connections 2024-03-16T20:46:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.127+0000 7fa5d8a8f640 1 --2- 172.21.15.122:0/2613418135 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fa5b4066e70 0x7fa5b4069330 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.127+0000 7fa5d8a8f640 1 --2- 172.21.15.122:0/2613418135 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fa5d4147080 0x7fa5d414a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.127+0000 7fa5d8a8f640 1 --2- 172.21.15.122:0/2613418135 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fa5d4140c70 0x7fa5d413f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.127+0000 7fa5d8a8f640 1 --2- 172.21.15.122:0/2613418135 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa5d40baca0 0x7fa5d41df720 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.127+0000 7fa5d8a8f640 1 -- 172.21.15.122:0/2613418135 >> 172.21.15.122:0/2613418135 conn(0x7fa5d406ee40 msgr2=0x7fa5d413c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.127+0000 7fa5d8a8f640 1 -- 172.21.15.122:0/2613418135 shutdown_connections 2024-03-16T20:46:00.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.127+0000 7fa5d8a8f640 1 -- 172.21.15.122:0/2613418135 wait complete. 2024-03-16T20:46:00.146 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:46:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.251+0000 7f717966d640 1 Processor -- start 2024-03-16T20:46:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.251+0000 7f717966d640 1 -- start start 2024-03-16T20:46:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f717966d640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f71740b4c50 0x7f71740b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f717966d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f7174144870 0x7f7174141290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f717966d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71740b1370 con 0x7f7174140e10 2024-03-16T20:46:00.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f717966d640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7174143ca0 con 0x7f7174144870 2024-03-16T20:46:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f717966d640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71740b4340 con 0x7f71740b4c50 2024-03-16T20:46:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f71727fc640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f7174140e10 0x7f7174144160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:40506/0 (socket says 172.21.15.122:40506) 2024-03-16T20:46:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f71727fc640 1 -- 172.21.15.122:0/3889817062 learned_addr learned my addr 172.21.15.122:0/3889817062 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:46:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f71737fe640 1 --2- 172.21.15.122:0/3889817062 >> v2:172.21.15.122:3300/0 conn(0x7f7174144870 0x7f7174141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f7172ffd640 1 --2- 172.21.15.122:0/3889817062 >> v2:172.21.15.195:3300/0 conn(0x7f71740b4c50 0x7f71740b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f7171ffb640 1 -- 172.21.15.122:0/3889817062 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 854379783 0 0) 0x7f71740b1370 con 0x7f7174140e10 2024-03-16T20:46:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f7171ffb640 1 -- 172.21.15.122:0/3889817062 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f715c003530 con 0x7f7174140e10 2024-03-16T20:46:00.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f71737fe640 1 -- 172.21.15.122:0/3889817062 >> v2:172.21.15.195:3300/0 conn(0x7f71740b4c50 msgr2=0x7f71740b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:00.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f71737fe640 1 --2- 172.21.15.122:0/3889817062 >> v2:172.21.15.195:3300/0 conn(0x7f71740b4c50 0x7f71740b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f71737fe640 1 -- 172.21.15.122:0/3889817062 >> v1:172.21.15.122:6789/0 conn(0x7f7174140e10 legacy=0x7f7174144160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.262 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f71737fe640 1 -- 172.21.15.122:0/3889817062 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71740b4620 con 0x7f7174144870 2024-03-16T20:46:00.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f71737fe640 1 --2- 172.21.15.122:0/3889817062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7174144870 0x7f7174141290 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f717406fc00 tx=0x7f716c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9dd307569beee4e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:00.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f7171ffb640 1 -- 172.21.15.122:0/3889817062 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c0a6660 con 0x7f7174144870 2024-03-16T20:46:00.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f7171ffb640 1 -- 172.21.15.122:0/3889817062 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716c0a67c0 con 0x7f7174144870 2024-03-16T20:46:00.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f7171ffb640 1 -- 172.21.15.122:0/3889817062 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c0a6b30 con 0x7f7174144870 2024-03-16T20:46:00.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f717966d640 1 -- 172.21.15.122:0/3889817062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7174144870 msgr2=0x7f7174141290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f717966d640 1 --2- 172.21.15.122:0/3889817062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7174144870 0x7f7174141290 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f717406fc00 tx=0x7f716c01c8a0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.267 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f717966d640 1 -- 172.21.15.122:0/3889817062 shutdown_connections 2024-03-16T20:46:00.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f717966d640 1 --2- 172.21.15.122:0/3889817062 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7174144870 0x7f7174141290 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f717966d640 1 --2- 172.21.15.122:0/3889817062 >> v2:172.21.15.195:3300/0 conn(0x7f71740b4c50 0x7f71740b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f717966d640 1 -- 172.21.15.122:0/3889817062 >> 172.21.15.122:0/3889817062 conn(0x7f71740b62a0 msgr2=0x7f71740b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:00.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f717966d640 1 -- 172.21.15.122:0/3889817062 shutdown_connections 2024-03-16T20:46:00.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.255+0000 7f717966d640 1 -- 172.21.15.122:0/3889817062 wait complete. 2024-03-16T20:46:00.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f717966d640 1 Processor -- start 2024-03-16T20:46:00.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f717966d640 1 -- start start 2024-03-16T20:46:00.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f717966d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71740b4c50 0x7f717413be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.268 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f717966d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7174140e10 0x7f717413cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f717966d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7174144870 0x7f71741e0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f717966d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7174148210 con 0x7f71740b4c50 2024-03-16T20:46:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f717966d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717413ba00 con 0x7f7174144870 2024-03-16T20:46:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f717966d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71740b3690 con 0x7f7174140e10 2024-03-16T20:46:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f7172ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71740b4c50 0x7f717413be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f7172ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71740b4c50 0x7f717413be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40674/0 (socket says 172.21.15.122:40674) 2024-03-16T20:46:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f7172ffd640 1 -- 172.21.15.122:0/2363496870 learned_addr learned my addr 172.21.15.122:0/2363496870 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f71737fe640 1 --2- 172.21.15.122:0/2363496870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7174144870 0x7f71741e0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.269 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f7172ffd640 1 -- 172.21.15.122:0/2363496870 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7174140e10 msgr2=0x7f717413cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f71727fc640 1 --2- 172.21.15.122:0/2363496870 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7174140e10 0x7f717413cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f7172ffd640 1 --2- 172.21.15.122:0/2363496870 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7174140e10 0x7f717413cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f7172ffd640 1 -- 172.21.15.122:0/2363496870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7174144870 msgr2=0x7f71741e0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f7172ffd640 1 --2- 172.21.15.122:0/2363496870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7174144870 0x7f71741e0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f7172ffd640 1 -- 172.21.15.122:0/2363496870 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c009a70 con 0x7f71740b4c50 2024-03-16T20:46:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f71737fe640 1 --2- 172.21.15.122:0/2363496870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7174144870 0x7f71741e0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f7172ffd640 1 --2- 172.21.15.122:0/2363496870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71740b4c50 0x7f717413be90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f716400d800 tx=0x7f716400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f715b7fe640 1 -- 172.21.15.122:0/2363496870 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716400b6c0 con 0x7f71740b4c50 2024-03-16T20:46:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f715b7fe640 1 -- 172.21.15.122:0/2363496870 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716400b820 con 0x7f71740b4c50 2024-03-16T20:46:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f717966d640 1 -- 172.21.15.122:0/2363496870 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717413b3b0 con 0x7f71740b4c50 2024-03-16T20:46:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f715b7fe640 1 -- 172.21.15.122:0/2363496870 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716400bb90 con 0x7f71740b4c50 2024-03-16T20:46:00.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f717966d640 1 -- 172.21.15.122:0/2363496870 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7174144e60 con 0x7f71740b4c50 2024-03-16T20:46:00.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.259+0000 7f717966d640 1 -- 172.21.15.122:0/2363496870 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7174075280 con 0x7f71740b4c50 2024-03-16T20:46:00.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.263+0000 7f715b7fe640 1 -- 172.21.15.122:0/2363496870 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f716400bcf0 con 0x7f71740b4c50 2024-03-16T20:46:00.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.263+0000 7f715b7fe640 1 --2- 172.21.15.122:0/2363496870 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7160066e40 0x7f7160069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.263+0000 7f715b7fe640 1 -- 172.21.15.122:0/2363496870 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7164084ff0 con 0x7f71740b4c50 2024-03-16T20:46:00.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.263+0000 7f71727fc640 1 --2- 172.21.15.122:0/2363496870 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7160066e40 0x7f7160069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.271 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.263+0000 7f71727fc640 1 --2- 172.21.15.122:0/2363496870 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7160066e40 0x7f7160069300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7168007fd0 tx=0x7f7168009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:00.272 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.266+0000 7f715b7fe640 1 -- 172.21.15.122:0/2363496870 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f7164051de0 con 0x7f71740b4c50 2024-03-16T20:46:00.426 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.418+0000 7f717966d640 1 -- 172.21.15.122:0/2363496870 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71741415b0 con 0x7f71740b4c50 2024-03-16T20:46:00.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.418+0000 7f715b7fe640 1 -- 172.21.15.122:0/2363496870 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+16705 (secure 0 0 0) 0x7f71640554a0 con 0x7f71740b4c50 2024-03-16T20:46:00.427 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:46:00.427 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:45.942613+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:42.678541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:43.274469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:46:00.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.422+0000 7f717966d640 1 -- 172.21.15.122:0/2363496870 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7160066e40 msgr2=0x7f7160069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.422+0000 7f717966d640 1 --2- 172.21.15.122:0/2363496870 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7160066e40 0x7f7160069300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7168007fd0 tx=0x7f7168009040 comp rx=0 tx=0).stop 2024-03-16T20:46:00.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.422+0000 7f717966d640 1 -- 172.21.15.122:0/2363496870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71740b4c50 msgr2=0x7f717413be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.422+0000 7f717966d640 1 --2- 172.21.15.122:0/2363496870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71740b4c50 0x7f717413be90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f716400d800 tx=0x7f716400dcc0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.422+0000 7f717966d640 1 -- 172.21.15.122:0/2363496870 shutdown_connections 2024-03-16T20:46:00.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.422+0000 7f717966d640 1 --2- 172.21.15.122:0/2363496870 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7160066e40 0x7f7160069300 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.422+0000 7f717966d640 1 --2- 172.21.15.122:0/2363496870 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7174144870 0x7f71741e0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.422+0000 7f717966d640 1 --2- 172.21.15.122:0/2363496870 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7174140e10 0x7f717413cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.422+0000 7f717966d640 1 --2- 172.21.15.122:0/2363496870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f71740b4c50 0x7f717413be90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.422+0000 7f717966d640 1 -- 172.21.15.122:0/2363496870 >> 172.21.15.122:0/2363496870 conn(0x7f71740b62a0 msgr2=0x7f71741365d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:00.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.422+0000 7f717966d640 1 -- 172.21.15.122:0/2363496870 shutdown_connections 2024-03-16T20:46:00.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.422+0000 7f717966d640 1 -- 172.21.15.122:0/2363496870 wait complete. 2024-03-16T20:46:00.446 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-16T20:46:00.446 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T20:46:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.550+0000 7f2428a07640 1 Processor -- start 2024-03-16T20:46:00.557 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.550+0000 7f2428a07640 1 -- start start 2024-03-16T20:46:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.550+0000 7f2428a07640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242413f3f0 0x7f2424150820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.550+0000 7f2428a07640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24240b1760 con 0x7f24240b64c0 2024-03-16T20:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.550+0000 7f2428a07640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24240b19b0 con 0x7f2424148f00 2024-03-16T20:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.550+0000 7f2428a07640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24240b3140 con 0x7f242413f3f0 2024-03-16T20:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242259c640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f24240b64c0 0x7f24241487f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:40518/0 (socket says 172.21.15.122:40518) 2024-03-16T20:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242259c640 1 -- 172.21.15.122:0/2806137415 learned_addr learned my addr 172.21.15.122:0/2806137415 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f2422d9d640 1 --2- 172.21.15.122:0/2806137415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242413f3f0 0x7f2424150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 -- 172.21.15.122:0/2806137415 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2222635348 0 0) 0x7f24240b1760 con 0x7f24240b64c0 2024-03-16T20:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 -- 172.21.15.122:0/2806137415 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2410003530 con 0x7f24240b64c0 2024-03-16T20:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 -- 172.21.15.122:0/2806137415 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 855044696 0 0) 0x7f2410003530 con 0x7f24240b64c0 2024-03-16T20:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 -- 172.21.15.122:0/2806137415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242413f3f0 msgr2=0x7f2424150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 --2- 172.21.15.122:0/2806137415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242413f3f0 0x7f2424150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.561 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 -- 172.21.15.122:0/2806137415 >> v1:172.21.15.195:6789/0 conn(0x7f2424148f00 legacy=0x7f242413ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 -- 172.21.15.122:0/2806137415 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24240b2ed0 con 0x7f24240b64c0 2024-03-16T20:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 -- 172.21.15.122:0/2806137415 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f2414001a00 con 0x7f24240b64c0 2024-03-16T20:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 -- 172.21.15.122:0/2806137415 >> v1:172.21.15.122:6789/0 conn(0x7f24240b64c0 legacy=0x7f24241487f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 --2- 172.21.15.122:0/2806137415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24100039b0 0x7f2410003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 --2- 172.21.15.122:0/2806137415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242413f3f0 0x7f24100047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 --2- 172.21.15.122:0/2806137415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2410004d10 0x7f2410005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 -- 172.21.15.122:0/2806137415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24240b3140 con 0x7f2410004d10 2024-03-16T20:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 -- 172.21.15.122:0/2806137415 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241000bb50 con 0x7f24100039b0 2024-03-16T20:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 -- 172.21.15.122:0/2806137415 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241000bd10 con 0x7f242413f3f0 2024-03-16T20:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f2422d9d640 1 --2- 172.21.15.122:0/2806137415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2410004d10 0x7f2410005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242259c640 1 --2- 172.21.15.122:0/2806137415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24100039b0 0x7f2410003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.562 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f2421d9b640 1 --2- 172.21.15.122:0/2806137415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242413f3f0 0x7f24100047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f2422d9d640 1 -- 172.21.15.122:0/2806137415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242413f3f0 msgr2=0x7f24100047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:00.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f2422d9d640 1 --2- 172.21.15.122:0/2806137415 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242413f3f0 0x7f24100047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f2422d9d640 1 -- 172.21.15.122:0/2806137415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24100039b0 msgr2=0x7f2410003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:00.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f2422d9d640 1 --2- 172.21.15.122:0/2806137415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24100039b0 0x7f2410003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f2422d9d640 1 -- 172.21.15.122:0/2806137415 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2410003530 con 0x7f2410004d10 2024-03-16T20:46:00.563 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242259c640 1 --2- 172.21.15.122:0/2806137415 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f24100039b0 0x7f2410003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:00.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f2422d9d640 1 --2- 172.21.15.122:0/2806137415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2410004d10 0x7f2410005190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f241c009ac0 tx=0x7f241c01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251eb1486f9801d5 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:00.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 -- 172.21.15.122:0/2806137415 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c0a5610 con 0x7f2410004d10 2024-03-16T20:46:00.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 -- 172.21.15.122:0/2806137415 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c0a5770 con 0x7f2410004d10 2024-03-16T20:46:00.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f242159a640 1 -- 172.21.15.122:0/2806137415 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c0a5b00 con 0x7f2410004d10 2024-03-16T20:46:00.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f2428a07640 1 -- 172.21.15.122:0/2806137415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2410004d10 msgr2=0x7f2410005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f2428a07640 1 --2- 172.21.15.122:0/2806137415 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2410004d10 0x7f2410005190 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f241c009ac0 tx=0x7f241c01ca20 comp rx=0 tx=0).stop 2024-03-16T20:46:00.567 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f2422d9d640 1 -- 172.21.15.122:0/2806137415 reap_dead start 2024-03-16T20:46:00.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f2428a07640 1 -- 172.21.15.122:0/2806137415 shutdown_connections 2024-03-16T20:46:00.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.554+0000 7f2428a07640 1 -- 172.21.15.122:0/2806137415 >> 172.21.15.122:0/2806137415 conn(0x7f24240b7350 msgr2=0x7f24240b7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:00.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2428a07640 1 -- 172.21.15.122:0/2806137415 shutdown_connections 2024-03-16T20:46:00.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2428a07640 1 -- 172.21.15.122:0/2806137415 wait complete. 2024-03-16T20:46:00.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2428a07640 1 Processor -- start 2024-03-16T20:46:00.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2428a07640 1 -- start start 2024-03-16T20:46:00.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2428a07640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2410004d10 0x7f242414dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2428a07640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2424147a40 0x7f24241e28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2428a07640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242412e580 0x7f242412ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2428a07640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2424133fd0 con 0x7f2424147a40 2024-03-16T20:46:00.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2428a07640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2424147230 con 0x7f2410004d10 2024-03-16T20:46:00.568 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2428a07640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2424146f50 con 0x7f242412e580 2024-03-16T20:46:00.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2421d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2424147a40 0x7f24241e28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2421d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2424147a40 0x7f24241e28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40702/0 (socket says 172.21.15.122:40702) 2024-03-16T20:46:00.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2421d9b640 1 -- 172.21.15.122:0/1133949151 learned_addr learned my addr 172.21.15.122:0/1133949151 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:00.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f242259c640 1 --2- 172.21.15.122:0/1133949151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2410004d10 0x7f242414dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2422d9d640 1 --2- 172.21.15.122:0/1133949151 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242412e580 0x7f242412ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2421d9b640 1 -- 172.21.15.122:0/1133949151 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242412e580 msgr2=0x7f242412ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2421d9b640 1 --2- 172.21.15.122:0/1133949151 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242412e580 0x7f242412ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2421d9b640 1 -- 172.21.15.122:0/1133949151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2410004d10 msgr2=0x7f242414dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2421d9b640 1 --2- 172.21.15.122:0/1133949151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2410004d10 0x7f242414dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2421d9b640 1 -- 172.21.15.122:0/1133949151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2424146070 con 0x7f2424147a40 2024-03-16T20:46:00.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2421d9b640 1 --2- 172.21.15.122:0/1133949151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2424147a40 0x7f24241e28b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f240c00d5a0 tx=0x7f240c00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:00.569 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f241affd640 1 -- 172.21.15.122:0/1133949151 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c005480 con 0x7f2424147a40 2024-03-16T20:46:00.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f241affd640 1 -- 172.21.15.122:0/1133949151 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240c003620 con 0x7f2424147a40 2024-03-16T20:46:00.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2428a07640 1 -- 172.21.15.122:0/1133949151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24241341d0 con 0x7f2424147a40 2024-03-16T20:46:00.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f241affd640 1 -- 172.21.15.122:0/1133949151 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c003990 con 0x7f2424147a40 2024-03-16T20:46:00.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2428a07640 1 -- 172.21.15.122:0/1133949151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2424147410 con 0x7f2424147a40 2024-03-16T20:46:00.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f2428a07640 1 -- 172.21.15.122:0/1133949151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24241462b0 con 0x7f2424147a40 2024-03-16T20:46:00.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.558+0000 7f241affd640 1 -- 172.21.15.122:0/1133949151 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f240c003af0 con 0x7f2424147a40 2024-03-16T20:46:00.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.562+0000 7f241affd640 1 --2- 172.21.15.122:0/1133949151 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2404066e40 0x7f2404069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.562+0000 7f241affd640 1 -- 172.21.15.122:0/1133949151 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f240c084860 con 0x7f2424147a40 2024-03-16T20:46:00.570 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.562+0000 7f242259c640 1 --2- 172.21.15.122:0/1133949151 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2404066e40 0x7f2404069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.571 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.562+0000 7f242259c640 1 --2- 172.21.15.122:0/1133949151 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2404066e40 0x7f2404069300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2414000f90 tx=0x7f2414009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:00.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.570+0000 7f241affd640 1 -- 172.21.15.122:0/1133949151 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f24241462b0 con 0x7f2424147a40 2024-03-16T20:46:00.760 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.754+0000 7f2428a07640 1 -- 172.21.15.122:0/1133949151 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2424075280 con 0x7f2424147a40 2024-03-16T20:46:00.761 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.754+0000 7f241affd640 1 -- 172.21.15.122:0/1133949151 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+3401 (secure 0 0 0) 0x7f240c0516e0 con 0x7f2424147a40 2024-03-16T20:46:00.761 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:46:00.761 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3638578490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3638578490},{"type":"v1","addr":"172.21.15.195:6835","nonce":3638578490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T20:45:44.280433+0000","modified":"2024-03-16T20:45:46.325382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4995":{"gid":4995,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6833/1841055069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1841055069},{"type":"v1","addr":"172.21.15.122:6833","nonce":1841055069}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4995,"qdb_cluster":[4995]},"id":2}]} 2024-03-16T20:46:00.762 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 13 2024-03-16T20:46:00.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.758+0000 7f2428a07640 1 -- 172.21.15.122:0/1133949151 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2404066e40 msgr2=0x7f2404069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.758+0000 7f2428a07640 1 --2- 172.21.15.122:0/1133949151 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2404066e40 0x7f2404069300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2414000f90 tx=0x7f2414009040 comp rx=0 tx=0).stop 2024-03-16T20:46:00.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.758+0000 7f2428a07640 1 -- 172.21.15.122:0/1133949151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2424147a40 msgr2=0x7f24241e28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.758+0000 7f2428a07640 1 --2- 172.21.15.122:0/1133949151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2424147a40 0x7f24241e28b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f240c00d5a0 tx=0x7f240c00da60 comp rx=0 tx=0).stop 2024-03-16T20:46:00.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.758+0000 7f2428a07640 1 -- 172.21.15.122:0/1133949151 shutdown_connections 2024-03-16T20:46:00.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.758+0000 7f2428a07640 1 --2- 172.21.15.122:0/1133949151 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2404066e40 0x7f2404069300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.758+0000 7f2428a07640 1 --2- 172.21.15.122:0/1133949151 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f242412e580 0x7f242412ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.758+0000 7f2428a07640 1 --2- 172.21.15.122:0/1133949151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2424147a40 0x7f24241e28b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.758+0000 7f2428a07640 1 --2- 172.21.15.122:0/1133949151 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2410004d10 0x7f242414dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.758+0000 7f2428a07640 1 -- 172.21.15.122:0/1133949151 >> 172.21.15.122:0/1133949151 conn(0x7f24240b7350 msgr2=0x7f24240b7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:00.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.758+0000 7f2428a07640 1 -- 172.21.15.122:0/1133949151 shutdown_connections 2024-03-16T20:46:00.765 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.758+0000 7f2428a07640 1 -- 172.21.15.122:0/1133949151 wait complete. 2024-03-16T20:46:00.779 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T20:46:00.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.878+0000 7f150ce06640 1 Processor -- start 2024-03-16T20:46:00.886 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.878+0000 7f150ce06640 1 -- start start 2024-03-16T20:46:00.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f150ce06640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1508140e10 0x7f1508141290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f150ce06640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1508144160 0x7f15081445f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f150ce06640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15080b5740 con 0x7f15080b4c50 2024-03-16T20:46:00.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f150ce06640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15080b1370 con 0x7f1508144160 2024-03-16T20:46:00.887 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f150ce06640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1508144b30 con 0x7f1508140e10 2024-03-16T20:46:00.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f1506d9d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1508144160 0x7f15081445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f1506d9d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1508144160 0x7f15081445f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40710/0 (socket says 172.21.15.122:40710) 2024-03-16T20:46:00.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f1506d9d640 1 -- 172.21.15.122:0/1309141285 learned_addr learned my addr 172.21.15.122:0/1309141285 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:00.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f14fffff640 1 --2- 172.21.15.122:0/1309141285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1508140e10 0x7f1508141290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f1506d9d640 1 -- 172.21.15.122:0/1309141285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1508140e10 msgr2=0x7f1508141290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:00.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f1506d9d640 1 --2- 172.21.15.122:0/1309141285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1508140e10 0x7f1508141290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f1506d9d640 1 -- 172.21.15.122:0/1309141285 >> v1:172.21.15.195:6789/0 conn(0x7f15080b4c50 legacy=0x7f15080b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:00.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f1506d9d640 1 -- 172.21.15.122:0/1309141285 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1508143ca0 con 0x7f1508144160 2024-03-16T20:46:00.888 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f14fffff640 1 --2- 172.21.15.122:0/1309141285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1508140e10 0x7f1508141290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:00.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f1506d9d640 1 --2- 172.21.15.122:0/1309141285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1508144160 0x7f15081445f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1500009e50 tx=0x7f150001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0bfc5473968b52 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:00.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f1505d9b640 1 -- 172.21.15.122:0/1309141285 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15000a55c0 con 0x7f1508144160 2024-03-16T20:46:00.889 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f1505d9b640 1 -- 172.21.15.122:0/1309141285 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15000a5720 con 0x7f1508144160 2024-03-16T20:46:00.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f1505d9b640 1 -- 172.21.15.122:0/1309141285 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15000a5ab0 con 0x7f1508144160 2024-03-16T20:46:00.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f150ce06640 1 -- 172.21.15.122:0/1309141285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1508144160 msgr2=0x7f15081445f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.890 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f150ce06640 1 --2- 172.21.15.122:0/1309141285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1508144160 0x7f15081445f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1500009e50 tx=0x7f150001c960 comp rx=0 tx=0).stop 2024-03-16T20:46:00.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f150ce06640 1 -- 172.21.15.122:0/1309141285 shutdown_connections 2024-03-16T20:46:00.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f150ce06640 1 --2- 172.21.15.122:0/1309141285 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1508144160 0x7f15081445f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f150ce06640 1 --2- 172.21.15.122:0/1309141285 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1508140e10 0x7f1508141290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f150ce06640 1 -- 172.21.15.122:0/1309141285 >> 172.21.15.122:0/1309141285 conn(0x7f15080b62a0 msgr2=0x7f15080b6680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:00.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.882+0000 7f150ce06640 1 -- 172.21.15.122:0/1309141285 shutdown_connections 2024-03-16T20:46:00.891 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f150ce06640 1 -- 172.21.15.122:0/1309141285 wait complete. 2024-03-16T20:46:00.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f150ce06640 1 Processor -- start 2024-03-16T20:46:00.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f150ce06640 1 -- start start 2024-03-16T20:46:00.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f150ce06640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15080b4c50 0x7f1508134f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f150ce06640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1508140e10 0x7f15081e0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f150ce06640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1508144160 0x7f15081e09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f150ce06640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150813b430 con 0x7f1508144160 2024-03-16T20:46:00.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f150ce06640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15081481b0 con 0x7f15080b4c50 2024-03-16T20:46:00.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f150ce06640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1508147f10 con 0x7f1508140e10 2024-03-16T20:46:00.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f1506d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1508144160 0x7f15081e09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f1506d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1508144160 0x7f15081e09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40712/0 (socket says 172.21.15.122:40712) 2024-03-16T20:46:00.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f1506d9d640 1 -- 172.21.15.122:0/2140530431 learned_addr learned my addr 172.21.15.122:0/2140530431 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:00.893 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f14fffff640 1 --2- 172.21.15.122:0/2140530431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1508140e10 0x7f15081e0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f1506d9d640 1 -- 172.21.15.122:0/2140530431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1508140e10 msgr2=0x7f15081e0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f150659c640 1 --2- 172.21.15.122:0/2140530431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15080b4c50 0x7f1508134f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f1506d9d640 1 --2- 172.21.15.122:0/2140530431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1508140e10 0x7f15081e0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f1506d9d640 1 -- 172.21.15.122:0/2140530431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15080b4c50 msgr2=0x7f1508134f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:00.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f1506d9d640 1 --2- 172.21.15.122:0/2140530431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15080b4c50 0x7f1508134f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:00.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f1506d9d640 1 -- 172.21.15.122:0/2140530431 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1500009b30 con 0x7f1508144160 2024-03-16T20:46:00.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f150659c640 1 --2- 172.21.15.122:0/2140530431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15080b4c50 0x7f1508134f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:00.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f1506d9d640 1 --2- 172.21.15.122:0/2140530431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1508144160 0x7f15081e09a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1500009e50 tx=0x7f1500019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:00.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f14feffd640 1 -- 172.21.15.122:0/2140530431 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15000b8670 con 0x7f1508144160 2024-03-16T20:46:00.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f14feffd640 1 -- 172.21.15.122:0/2140530431 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15000b87d0 con 0x7f1508144160 2024-03-16T20:46:00.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f150ce06640 1 -- 172.21.15.122:0/2140530431 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150813b090 con 0x7f1508144160 2024-03-16T20:46:00.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f14feffd640 1 -- 172.21.15.122:0/2140530431 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15000b8b40 con 0x7f1508144160 2024-03-16T20:46:00.894 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f150ce06640 1 -- 172.21.15.122:0/2140530431 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1508147250 con 0x7f1508144160 2024-03-16T20:46:00.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.886+0000 7f150ce06640 1 -- 172.21.15.122:0/2140530431 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1508075280 con 0x7f1508144160 2024-03-16T20:46:00.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.890+0000 7f14feffd640 1 -- 172.21.15.122:0/2140530431 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f15000b8ca0 con 0x7f1508144160 2024-03-16T20:46:00.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.890+0000 7f14feffd640 1 --2- 172.21.15.122:0/2140530431 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f14e8066e70 0x7f14e8069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:00.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.890+0000 7f14feffd640 1 -- 172.21.15.122:0/2140530431 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f150011fc20 con 0x7f1508144160 2024-03-16T20:46:00.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.890+0000 7f150659c640 1 --2- 172.21.15.122:0/2140530431 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f14e8066e70 0x7f14e8069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:00.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.890+0000 7f150659c640 1 --2- 172.21.15.122:0/2140530431 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f14e8066e70 0x7f14e8069330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1508135750 tx=0x7f14f8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:00.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:00.894+0000 7f14feffd640 1 -- 172.21.15.122:0/2140530431 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f15000ecbc0 con 0x7f1508144160 2024-03-16T20:46:01.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.078+0000 7f150ce06640 1 -- 172.21.15.122:0/2140530431 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f150813a330 con 0x7f1508144160 2024-03-16T20:46:01.086 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.082+0000 7f14feffd640 1 -- 172.21.15.122:0/2140530431 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+3401 (secure 0 0 0) 0x7f15000f0280 con 0x7f1508144160 2024-03-16T20:46:01.086 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:46:01.087 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3638578490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3638578490},{"type":"v1","addr":"172.21.15.195:6835","nonce":3638578490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-03-16T20:45:44.280433+0000","modified":"2024-03-16T20:45:46.325382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4995},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4995":{"gid":4995,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.122:6833/1841055069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":1841055069},{"type":"v1","addr":"172.21.15.122:6833","nonce":1841055069}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4995,"qdb_cluster":[4995]},"id":2}]} 2024-03-16T20:46:01.087 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 13 2024-03-16T20:46:01.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.082+0000 7f150ce06640 1 -- 172.21.15.122:0/2140530431 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f14e8066e70 msgr2=0x7f14e8069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.082+0000 7f150ce06640 1 --2- 172.21.15.122:0/2140530431 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f14e8066e70 0x7f14e8069330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1508135750 tx=0x7f14f8009040 comp rx=0 tx=0).stop 2024-03-16T20:46:01.088 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.082+0000 7f150ce06640 1 -- 172.21.15.122:0/2140530431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1508144160 msgr2=0x7f15081e09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.082+0000 7f150ce06640 1 --2- 172.21.15.122:0/2140530431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1508144160 0x7f15081e09a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f1500009e50 tx=0x7f1500019700 comp rx=0 tx=0).stop 2024-03-16T20:46:01.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.082+0000 7f150ce06640 1 -- 172.21.15.122:0/2140530431 shutdown_connections 2024-03-16T20:46:01.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.082+0000 7f150ce06640 1 --2- 172.21.15.122:0/2140530431 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f14e8066e70 0x7f14e8069330 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.082+0000 7f150ce06640 1 --2- 172.21.15.122:0/2140530431 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1508144160 0x7f15081e09a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.082+0000 7f150ce06640 1 --2- 172.21.15.122:0/2140530431 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1508140e10 0x7f15081e0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.082+0000 7f150ce06640 1 --2- 172.21.15.122:0/2140530431 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f15080b4c50 0x7f1508134f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.082+0000 7f150ce06640 1 -- 172.21.15.122:0/2140530431 >> 172.21.15.122:0/2140530431 conn(0x7f15080b62a0 msgr2=0x7f1508070da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:01.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.082+0000 7f150ce06640 1 -- 172.21.15.122:0/2140530431 shutdown_connections 2024-03-16T20:46:01.089 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.082+0000 7f150ce06640 1 -- 172.21.15.122:0/2140530431 wait complete. 2024-03-16T20:46:01.103 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:46:01.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.210+0000 7f2bba772640 1 Processor -- start 2024-03-16T20:46:01.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.210+0000 7f2bba772640 1 -- start start 2024-03-16T20:46:01.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.214+0000 7f2bba772640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f2bb4066fa0 0x7f2bb40af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:01.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.214+0000 7f2bba772640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb40b4ef0 con 0x7f2bb40b3b50 2024-03-16T20:46:01.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.214+0000 7f2bba772640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb40ae7e0 con 0x7f2bb4072ef0 2024-03-16T20:46:01.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.214+0000 7f2bba772640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb40afda0 con 0x7f2bb4066fa0 2024-03-16T20:46:01.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.214+0000 7f2bb3fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f2bb4066fa0 0x7f2bb40af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:01.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.214+0000 7f2bb3fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f2bb4066fa0 0x7f2bb40af740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40716/0 (socket says 172.21.15.122:40716) 2024-03-16T20:46:01.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.214+0000 7f2bb3fff640 1 -- 172.21.15.122:0/1928629520 learned_addr learned my addr 172.21.15.122:0/1928629520 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:01.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.214+0000 7f2bb3fff640 1 -- 172.21.15.122:0/1928629520 >> v1:172.21.15.195:6789/0 conn(0x7f2bb4072ef0 legacy=0x7f2bb4151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:01.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.214+0000 7f2bb3fff640 1 -- 172.21.15.122:0/1928629520 >> v1:172.21.15.122:6789/0 conn(0x7f2bb40b3b50 legacy=0x7f2bb40727e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.214+0000 7f2bb3fff640 1 -- 172.21.15.122:0/1928629520 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb40ae210 con 0x7f2bb4066fa0 2024-03-16T20:46:01.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.214+0000 7f2bb3fff640 1 --2- 172.21.15.122:0/1928629520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bb4066fa0 0x7f2bb40af740 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009870 tx=0x7f2ba401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5195695e8a1d2b19 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:01.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.214+0000 7f2bb37fe640 1 -- 172.21.15.122:0/1928629520 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba40ab600 con 0x7f2bb4066fa0 2024-03-16T20:46:01.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.214+0000 7f2bb37fe640 1 -- 172.21.15.122:0/1928629520 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba40a7050 con 0x7f2bb4066fa0 2024-03-16T20:46:01.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.214+0000 7f2bb37fe640 1 -- 172.21.15.122:0/1928629520 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba40ab980 con 0x7f2bb4066fa0 2024-03-16T20:46:01.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2bba772640 1 -- 172.21.15.122:0/1928629520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bb4066fa0 msgr2=0x7f2bb40af740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2bba772640 1 --2- 172.21.15.122:0/1928629520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bb4066fa0 0x7f2bb40af740 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009870 tx=0x7f2ba401c3d0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2bba772640 1 -- 172.21.15.122:0/1928629520 shutdown_connections 2024-03-16T20:46:01.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2bba772640 1 --2- 172.21.15.122:0/1928629520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bb4066fa0 0x7f2bb40af740 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2bba772640 1 -- 172.21.15.122:0/1928629520 >> 172.21.15.122:0/1928629520 conn(0x7f2bb406d350 msgr2=0x7f2bb406d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:01.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2bba772640 1 -- 172.21.15.122:0/1928629520 shutdown_connections 2024-03-16T20:46:01.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2bba772640 1 -- 172.21.15.122:0/1928629520 wait complete. 2024-03-16T20:46:01.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2bba772640 1 Processor -- start 2024-03-16T20:46:01.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2bba772640 1 -- start start 2024-03-16T20:46:01.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2bba772640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bb4066fa0 0x7f2bb40b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:01.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2bba772640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bb40b3b50 0x7f2bb40b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:01.226 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2bba772640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bb4145290 0x7f2bb40b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:01.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2bba772640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb413fb30 con 0x7f2bb40b3b50 2024-03-16T20:46:01.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2bba772640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb413f4e0 con 0x7f2bb4145290 2024-03-16T20:46:01.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2bba772640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb413f7c0 con 0x7f2bb4066fa0 2024-03-16T20:46:01.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2babfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bb40b3b50 0x7f2bb40b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:01.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2babfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bb40b3b50 0x7f2bb40b06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40718/0 (socket says 172.21.15.122:40718) 2024-03-16T20:46:01.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2babfff640 1 -- 172.21.15.122:0/1213790028 learned_addr learned my addr 172.21.15.122:0/1213790028 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:01.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2bb8d0f640 1 --2- 172.21.15.122:0/1213790028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bb4145290 0x7f2bb40b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:01.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2bb3fff640 1 --2- 172.21.15.122:0/1213790028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bb4066fa0 0x7f2bb40b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:01.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2babfff640 1 -- 172.21.15.122:0/1213790028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bb4066fa0 msgr2=0x7f2bb40b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2babfff640 1 --2- 172.21.15.122:0/1213790028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bb4066fa0 0x7f2bb40b3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2babfff640 1 -- 172.21.15.122:0/1213790028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bb4145290 msgr2=0x7f2bb40b0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2babfff640 1 --2- 172.21.15.122:0/1213790028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bb4145290 0x7f2bb40b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2babfff640 1 -- 172.21.15.122:0/1213790028 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba40094d0 con 0x7f2bb40b3b50 2024-03-16T20:46:01.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.218+0000 7f2babfff640 1 --2- 172.21.15.122:0/1213790028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bb40b3b50 0x7f2bb40b06c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00c370 tx=0x7f2b9c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:01.227 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.222+0000 7f2bb17fa640 1 -- 172.21.15.122:0/1213790028 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c0052d0 con 0x7f2bb40b3b50 2024-03-16T20:46:01.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.222+0000 7f2bb17fa640 1 -- 172.21.15.122:0/1213790028 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b9c005430 con 0x7f2bb40b3b50 2024-03-16T20:46:01.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.222+0000 7f2bba772640 1 -- 172.21.15.122:0/1213790028 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb40af530 con 0x7f2bb40b3b50 2024-03-16T20:46:01.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.222+0000 7f2bb17fa640 1 -- 172.21.15.122:0/1213790028 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c007260 con 0x7f2bb40b3b50 2024-03-16T20:46:01.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.222+0000 7f2bba772640 1 -- 172.21.15.122:0/1213790028 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb41403f0 con 0x7f2bb40b3b50 2024-03-16T20:46:01.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.222+0000 7f2bba772640 1 -- 172.21.15.122:0/1213790028 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb409bca0 con 0x7f2bb40b3b50 2024-03-16T20:46:01.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.222+0000 7f2bb17fa640 1 -- 172.21.15.122:0/1213790028 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f2b9c0073c0 con 0x7f2bb40b3b50 2024-03-16T20:46:01.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.222+0000 7f2bb17fa640 1 --2- 172.21.15.122:0/1213790028 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2b94066e40 0x7f2b94069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:01.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.222+0000 7f2bb17fa640 1 -- 172.21.15.122:0/1213790028 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b9c083b80 con 0x7f2bb40b3b50 2024-03-16T20:46:01.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.222+0000 7f2bb3fff640 1 --2- 172.21.15.122:0/1213790028 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2b94066e40 0x7f2b94069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:01.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.222+0000 7f2bb3fff640 1 --2- 172.21.15.122:0/1213790028 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2b94066e40 0x7f2b94069300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40ad000 tx=0x7f2ba4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:01.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.226+0000 7f2bb17fa640 1 -- 172.21.15.122:0/1213790028 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2b9c050a00 con 0x7f2bb40b3b50 2024-03-16T20:46:01.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.374+0000 7f2bba772640 1 -- 172.21.15.122:0/1213790028 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2bb40b4ed0 con 0x7f2bb40b3b50 2024-03-16T20:46:01.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.378+0000 7f2bb17fa640 1 -- 172.21.15.122:0/1213790028 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+16705 (secure 0 0 0) 0x7f2b9c0540c0 con 0x7f2bb40b3b50 2024-03-16T20:46:01.383 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:46:01.383 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":30,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:45:45.942613+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-03-16T20:45:42.678541+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-03-16T20:45:43.274469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:46:01.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.378+0000 7f2bba772640 1 -- 172.21.15.122:0/1213790028 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2b94066e40 msgr2=0x7f2b94069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.378+0000 7f2bba772640 1 --2- 172.21.15.122:0/1213790028 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2b94066e40 0x7f2b94069300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40ad000 tx=0x7f2ba4008040 comp rx=0 tx=0).stop 2024-03-16T20:46:01.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.378+0000 7f2bba772640 1 -- 172.21.15.122:0/1213790028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bb40b3b50 msgr2=0x7f2bb40b06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.378+0000 7f2bba772640 1 --2- 172.21.15.122:0/1213790028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bb40b3b50 0x7f2bb40b06c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c00c370 tx=0x7f2b9c00c830 comp rx=0 tx=0).stop 2024-03-16T20:46:01.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.378+0000 7f2bba772640 1 -- 172.21.15.122:0/1213790028 shutdown_connections 2024-03-16T20:46:01.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.378+0000 7f2bba772640 1 --2- 172.21.15.122:0/1213790028 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2b94066e40 0x7f2b94069300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.378+0000 7f2bba772640 1 --2- 172.21.15.122:0/1213790028 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2bb4145290 0x7f2bb40b0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.378+0000 7f2bba772640 1 --2- 172.21.15.122:0/1213790028 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2bb40b3b50 0x7f2bb40b06c0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.378+0000 7f2bba772640 1 --2- 172.21.15.122:0/1213790028 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2bb4066fa0 0x7f2bb40b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.378+0000 7f2bba772640 1 -- 172.21.15.122:0/1213790028 >> 172.21.15.122:0/1213790028 conn(0x7f2bb406d350 msgr2=0x7f2bb406d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:01.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.378+0000 7f2bba772640 1 -- 172.21.15.122:0/1213790028 shutdown_connections 2024-03-16T20:46:01.385 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.378+0000 7f2bba772640 1 -- 172.21.15.122:0/1213790028 wait complete. 2024-03-16T20:46:01.400 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-16T20:46:01.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.502+0000 7f97b96c9640 1 Processor -- start 2024-03-16T20:46:01.508 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.502+0000 7f97b96c9640 1 -- start start 2024-03-16T20:46:01.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.502+0000 7f97b96c9640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f97b40b7390 0x7f97b40b6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:01.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.502+0000 7f97b96c9640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f97b4147900 0x7f97b41519b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:01.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.502+0000 7f97b96c9640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b406de40 con 0x7f97b40b6a60 2024-03-16T20:46:01.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.502+0000 7f97b96c9640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b4070ac0 con 0x7f97b40b7390 2024-03-16T20:46:01.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.502+0000 7f97b96c9640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b406c4d0 con 0x7f97b4147900 2024-03-16T20:46:01.510 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.502+0000 7f97b2ffd640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f97b40b7390 0x7f97b40b6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:01.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.502+0000 7f97aa7fc640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f97b40b6a60 0x7f97b41471f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:40538/0 (socket says 172.21.15.122:40538) 2024-03-16T20:46:01.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.502+0000 7f97aa7fc640 1 -- 172.21.15.122:0/202893277 learned_addr learned my addr 172.21.15.122:0/202893277 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:46:01.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.502+0000 7f97b37fe640 1 --2- 172.21.15.122:0/202893277 >> v2:172.21.15.195:3300/0 conn(0x7f97b4147900 0x7f97b41519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:01.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.502+0000 7f97b2ffd640 1 -- 172.21.15.122:0/202893277 >> v2:172.21.15.195:3300/0 conn(0x7f97b4147900 msgr2=0x7f97b41519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:01.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.502+0000 7f97b2ffd640 1 --2- 172.21.15.122:0/202893277 >> v2:172.21.15.195:3300/0 conn(0x7f97b4147900 0x7f97b41519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.502+0000 7f97b2ffd640 1 -- 172.21.15.122:0/202893277 >> v1:172.21.15.122:6789/0 conn(0x7f97b40b6a60 legacy=0x7f97b41471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.502+0000 7f97b2ffd640 1 -- 172.21.15.122:0/202893277 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b406c0a0 con 0x7f97b40b7390 2024-03-16T20:46:01.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.502+0000 7f97b27fc640 1 -- 172.21.15.122:0/202893277 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153817249 0 0) 0x7f97b406de40 con 0x7f97b40b6a60 2024-03-16T20:46:01.511 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.502+0000 7f97b37fe640 1 --2- 172.21.15.122:0/202893277 >> v2:172.21.15.195:3300/0 conn(0x7f97b4147900 0x7f97b41519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:01.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b2ffd640 1 --2- 172.21.15.122:0/202893277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b40b7390 0x7f97b40b6520 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f97b4072e10 tx=0x7f97a401c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61a9f39f32cc7562 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:01.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b27fc640 1 -- 172.21.15.122:0/202893277 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a40ab600 con 0x7f97b40b7390 2024-03-16T20:46:01.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b27fc640 1 -- 172.21.15.122:0/202893277 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a40a7050 con 0x7f97b40b7390 2024-03-16T20:46:01.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b27fc640 1 -- 172.21.15.122:0/202893277 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a40ab980 con 0x7f97b40b7390 2024-03-16T20:46:01.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b96c9640 1 -- 172.21.15.122:0/202893277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b40b7390 msgr2=0x7f97b40b6520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b96c9640 1 --2- 172.21.15.122:0/202893277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b40b7390 0x7f97b40b6520 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f97b4072e10 tx=0x7f97a401c2a0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b96c9640 1 -- 172.21.15.122:0/202893277 shutdown_connections 2024-03-16T20:46:01.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b96c9640 1 --2- 172.21.15.122:0/202893277 >> v2:172.21.15.195:3300/0 conn(0x7f97b4147900 0x7f97b41519b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b96c9640 1 --2- 172.21.15.122:0/202893277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b40b7390 0x7f97b40b6520 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.513 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b96c9640 1 -- 172.21.15.122:0/202893277 >> 172.21.15.122:0/202893277 conn(0x7f97b406fc00 msgr2=0x7f97b4070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:01.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b96c9640 1 -- 172.21.15.122:0/202893277 shutdown_connections 2024-03-16T20:46:01.514 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b96c9640 1 -- 172.21.15.122:0/202893277 wait complete. 2024-03-16T20:46:01.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b96c9640 1 Processor -- start 2024-03-16T20:46:01.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b96c9640 1 -- start start 2024-03-16T20:46:01.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b96c9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97b40b6a60 0x7f97b41463c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:01.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b96c9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97b40b7390 0x7f97b41df500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:01.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b96c9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b4147900 0x7f97b41dfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:01.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b96c9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b41412b0 con 0x7f97b4147900 2024-03-16T20:46:01.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b96c9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b413fbe0 con 0x7f97b40b6a60 2024-03-16T20:46:01.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.506+0000 7f97b96c9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b41414c0 con 0x7f97b40b7390 2024-03-16T20:46:01.516 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97b37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b4147900 0x7f97b41dfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:01.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97b37fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b4147900 0x7f97b41dfa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40724/0 (socket says 172.21.15.122:40724) 2024-03-16T20:46:01.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97b37fe640 1 -- 172.21.15.122:0/3065705752 learned_addr learned my addr 172.21.15.122:0/3065705752 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:01.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97b2ffd640 1 --2- 172.21.15.122:0/3065705752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97b40b6a60 0x7f97b41463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:01.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97b37fe640 1 -- 172.21.15.122:0/3065705752 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97b40b7390 msgr2=0x7f97b41df500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97b37fe640 1 --2- 172.21.15.122:0/3065705752 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97b40b7390 0x7f97b41df500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97b37fe640 1 -- 172.21.15.122:0/3065705752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97b40b6a60 msgr2=0x7f97b41463c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97b37fe640 1 --2- 172.21.15.122:0/3065705752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97b40b6a60 0x7f97b41463c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97b37fe640 1 -- 172.21.15.122:0/3065705752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a4009420 con 0x7f97b4147900 2024-03-16T20:46:01.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97b37fe640 1 --2- 172.21.15.122:0/3065705752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b4147900 0x7f97b41dfa40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00cf50 tx=0x7f97ac00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:01.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97abfff640 1 -- 172.21.15.122:0/3065705752 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac00d840 con 0x7f97b4147900 2024-03-16T20:46:01.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97abfff640 1 -- 172.21.15.122:0/3065705752 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ac00d9a0 con 0x7f97b4147900 2024-03-16T20:46:01.517 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97b96c9640 1 -- 172.21.15.122:0/3065705752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b4140e20 con 0x7f97b4147900 2024-03-16T20:46:01.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97abfff640 1 -- 172.21.15.122:0/3065705752 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac00dd50 con 0x7f97b4147900 2024-03-16T20:46:01.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97b96c9640 1 -- 172.21.15.122:0/3065705752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b41470e0 con 0x7f97b4147900 2024-03-16T20:46:01.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97b96c9640 1 -- 172.21.15.122:0/3065705752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b4140bf0 con 0x7f97b4147900 2024-03-16T20:46:01.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97abfff640 1 -- 172.21.15.122:0/3065705752 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f97ac01d360 con 0x7f97b4147900 2024-03-16T20:46:01.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97abfff640 1 --2- 172.21.15.122:0/3065705752 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9794066e40 0x7f9794069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:01.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97abfff640 1 -- 172.21.15.122:0/3065705752 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97ac085840 con 0x7f97b4147900 2024-03-16T20:46:01.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97b2ffd640 1 --2- 172.21.15.122:0/3065705752 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9794066e40 0x7f9794069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:01.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.510+0000 7f97b2ffd640 1 --2- 172.21.15.122:0/3065705752 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9794066e40 0x7f9794069300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f97a40ad000 tx=0x7f97a4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:01.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.514+0000 7f97abfff640 1 -- 172.21.15.122:0/3065705752 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f97ac0527a0 con 0x7f97b4147900 2024-03-16T20:46:01.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.694+0000 7f97b96c9640 1 -- 172.21.15.122:0/3065705752 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f97b40b1920 con 0x7f97b4147900 2024-03-16T20:46:01.700 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-03-16T20:46:01.694+0000 7f11e71dc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-16T20:46:01.702 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.698+0000 7f97abfff640 1 -- 172.21.15.122:0/3065705752 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v14) v1 ==== 161+0+0 (secure 0 0 0) 0x7f97ac01a090 con 0x7f97b4147900 2024-03-16T20:46:01.702 INFO:teuthology.orchestra.run.smithi122.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-16T20:46:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.702+0000 7f97b96c9640 1 -- 172.21.15.122:0/3065705752 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9794066e40 msgr2=0x7f9794069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.702+0000 7f97b96c9640 1 --2- 172.21.15.122:0/3065705752 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9794066e40 0x7f9794069300 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f97a40ad000 tx=0x7f97a4008040 comp rx=0 tx=0).stop 2024-03-16T20:46:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.702+0000 7f97b96c9640 1 -- 172.21.15.122:0/3065705752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b4147900 msgr2=0x7f97b41dfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.702+0000 7f97b96c9640 1 --2- 172.21.15.122:0/3065705752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b4147900 0x7f97b41dfa40 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00cf50 tx=0x7f97ac00a7d0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.702+0000 7f97b96c9640 1 -- 172.21.15.122:0/3065705752 shutdown_connections 2024-03-16T20:46:01.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.702+0000 7f97b96c9640 1 --2- 172.21.15.122:0/3065705752 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9794066e40 0x7f9794069300 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.702+0000 7f97b96c9640 1 --2- 172.21.15.122:0/3065705752 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f97b4147900 0x7f97b41dfa40 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.702+0000 7f97b96c9640 1 --2- 172.21.15.122:0/3065705752 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f97b40b7390 0x7f97b41df500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.702+0000 7f97b96c9640 1 --2- 172.21.15.122:0/3065705752 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f97b40b6a60 0x7f97b41463c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.702+0000 7f97b96c9640 1 -- 172.21.15.122:0/3065705752 >> 172.21.15.122:0/3065705752 conn(0x7f97b406fc00 msgr2=0x7f97b40b9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.702+0000 7f97b96c9640 1 -- 172.21.15.122:0/3065705752 shutdown_connections 2024-03-16T20:46:01.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.702+0000 7f97b96c9640 1 -- 172.21.15.122:0/3065705752 wait complete. 2024-03-16T20:46:01.708 INFO:tasks.ceph.mds.a.smithi122.stderr: -2320> 2024-03-16T20:45:48.887+0000 7fe85768f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-16T20:46:01.724 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-16T20:46:01.736 INFO:tasks.ceph.mds.a.smithi122.stdout:starting mds.a at 2024-03-16T20:46:01.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.830+0000 7f3ecf60c640 1 Processor -- start 2024-03-16T20:46:01.836 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.830+0000 7f3ecf60c640 1 -- start start 2024-03-16T20:46:01.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.830+0000 7f3ecf60c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f3ec806e030 0x7f3ec8144fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:01.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.830+0000 7f3ecf60c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ec813f4a0 0x7f3ec814f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:01.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.830+0000 7f3ecf60c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ec809a130 con 0x7f3ec8145510 2024-03-16T20:46:01.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.830+0000 7f3ecf60c640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ec806e410 con 0x7f3ec806e030 2024-03-16T20:46:01.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.830+0000 7f3ecf60c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec8099de0 con 0x7f3ec813f4a0 2024-03-16T20:46:01.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.830+0000 7f3ecd3a8640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f3ec806e030 0x7f3ec8144fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:01.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.830+0000 7f3eccba7640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f3ec8145510 0x7f3ec813ed90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:40564/0 (socket says 172.21.15.122:40564) 2024-03-16T20:46:01.838 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.830+0000 7f3eccba7640 1 -- 172.21.15.122:0/3708305453 learned_addr learned my addr 172.21.15.122:0/3708305453 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:46:01.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.830+0000 7f3ecdba9640 1 --2- 172.21.15.122:0/3708305453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ec813f4a0 0x7f3ec814f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:01.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.830+0000 7f3ecd3a8640 1 -- 172.21.15.122:0/3708305453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ec813f4a0 msgr2=0x7f3ec814f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:01.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.830+0000 7f3ecd3a8640 1 --2- 172.21.15.122:0/3708305453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ec813f4a0 0x7f3ec814f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.830+0000 7f3ecd3a8640 1 -- 172.21.15.122:0/3708305453 >> v1:172.21.15.122:6789/0 conn(0x7f3ec8145510 legacy=0x7f3ec813ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.830+0000 7f3ebffff640 1 -- 172.21.15.122:0/3708305453 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228832753 0 0) 0x7f3ec809a130 con 0x7f3ec8145510 2024-03-16T20:46:01.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.830+0000 7f3ecd3a8640 1 -- 172.21.15.122:0/3708305453 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ec806c620 con 0x7f3ec806e030 2024-03-16T20:46:01.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.830+0000 7f3ecdba9640 1 --2- 172.21.15.122:0/3708305453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ec813f4a0 0x7f3ec814f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:01.840 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.834+0000 7f3ecd3a8640 1 --2- 172.21.15.122:0/3708305453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ec806e030 0x7f3ec8144fd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3eb80097f0 tx=0x7f3eb801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d925c09117c6b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.834+0000 7f3ebffff640 1 -- 172.21.15.122:0/3708305453 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb80ab600 con 0x7f3ec806e030 2024-03-16T20:46:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.834+0000 7f3ebffff640 1 -- 172.21.15.122:0/3708305453 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eb80a7050 con 0x7f3ec806e030 2024-03-16T20:46:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.834+0000 7f3ebffff640 1 -- 172.21.15.122:0/3708305453 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb80ab980 con 0x7f3ec806e030 2024-03-16T20:46:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.834+0000 7f3ecf60c640 1 -- 172.21.15.122:0/3708305453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ec806e030 msgr2=0x7f3ec8144fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.834+0000 7f3ecf60c640 1 --2- 172.21.15.122:0/3708305453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ec806e030 0x7f3ec8144fd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3eb80097f0 tx=0x7f3eb801c410 comp rx=0 tx=0).stop 2024-03-16T20:46:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.834+0000 7f3ecf60c640 1 -- 172.21.15.122:0/3708305453 shutdown_connections 2024-03-16T20:46:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.834+0000 7f3ecf60c640 1 --2- 172.21.15.122:0/3708305453 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ec813f4a0 0x7f3ec814f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.834+0000 7f3ecf60c640 1 --2- 172.21.15.122:0/3708305453 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ec806e030 0x7f3ec8144fd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.834+0000 7f3ecf60c640 1 -- 172.21.15.122:0/3708305453 >> 172.21.15.122:0/3708305453 conn(0x7f3ec806d2b0 msgr2=0x7f3ec806d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.834+0000 7f3ecf60c640 1 -- 172.21.15.122:0/3708305453 shutdown_connections 2024-03-16T20:46:01.841 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.834+0000 7f3ecf60c640 1 -- 172.21.15.122:0/3708305453 wait complete. 2024-03-16T20:46:01.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.834+0000 7f3ecf60c640 1 Processor -- start 2024-03-16T20:46:01.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.834+0000 7f3ecf60c640 1 -- start start 2024-03-16T20:46:01.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ecf60c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ec806e030 0x7f3ec8139010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:01.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ecf60c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ec813f4a0 0x7f3ec80af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:01.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ecf60c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ec8145510 0x7f3ec8132080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:01.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ecf60c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec8071ac0 con 0x7f3ec813f4a0 2024-03-16T20:46:01.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ecf60c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec8071380 con 0x7f3ec8145510 2024-03-16T20:46:01.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ecf60c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec8071d20 con 0x7f3ec806e030 2024-03-16T20:46:01.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3eccba7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ec813f4a0 0x7f3ec80af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:01.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3eccba7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ec813f4a0 0x7f3ec80af440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40746/0 (socket says 172.21.15.122:40746) 2024-03-16T20:46:01.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3eccba7640 1 -- 172.21.15.122:0/2791739703 learned_addr learned my addr 172.21.15.122:0/2791739703 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:01.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3eccba7640 1 -- 172.21.15.122:0/2791739703 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ec806e030 msgr2=0x7f3ec8139010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3eccba7640 1 --2- 172.21.15.122:0/2791739703 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ec806e030 0x7f3ec8139010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3eccba7640 1 -- 172.21.15.122:0/2791739703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ec8145510 msgr2=0x7f3ec8132080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ecdba9640 1 --2- 172.21.15.122:0/2791739703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ec8145510 0x7f3ec8132080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3eccba7640 1 --2- 172.21.15.122:0/2791739703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ec8145510 0x7f3ec8132080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3eccba7640 1 -- 172.21.15.122:0/2791739703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb8009420 con 0x7f3ec813f4a0 2024-03-16T20:46:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3eccba7640 1 --2- 172.21.15.122:0/2791739703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ec813f4a0 0x7f3ec80af440 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400c450 tx=0x7f3ec400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ebdffb640 1 -- 172.21.15.122:0/2791739703 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec40074e0 con 0x7f3ec813f4a0 2024-03-16T20:46:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ebdffb640 1 -- 172.21.15.122:0/2791739703 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec4007640 con 0x7f3ec813f4a0 2024-03-16T20:46:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ecf60c640 1 -- 172.21.15.122:0/2791739703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec8072000 con 0x7f3ec813f4a0 2024-03-16T20:46:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ebdffb640 1 -- 172.21.15.122:0/2791739703 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec40049f0 con 0x7f3ec813f4a0 2024-03-16T20:46:01.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ecf60c640 1 -- 172.21.15.122:0/2791739703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec8072520 con 0x7f3ec813f4a0 2024-03-16T20:46:01.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ecf60c640 1 -- 172.21.15.122:0/2791739703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec8074bb0 con 0x7f3ec813f4a0 2024-03-16T20:46:01.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ebdffb640 1 -- 172.21.15.122:0/2791739703 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f3ec4004b50 con 0x7f3ec813f4a0 2024-03-16T20:46:01.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ebdffb640 1 --2- 172.21.15.122:0/2791739703 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3eac066e40 0x7f3eac069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:01.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ebdffb640 1 -- 172.21.15.122:0/2791739703 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3ec4083f70 con 0x7f3ec813f4a0 2024-03-16T20:46:01.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ecd3a8640 1 --2- 172.21.15.122:0/2791739703 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3eac066e40 0x7f3eac069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:01.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.838+0000 7f3ecd3a8640 1 --2- 172.21.15.122:0/2791739703 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3eac066e40 0x7f3eac069300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3eb80ad000 tx=0x7f3eb8004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:01.849 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:01.842+0000 7f3ebdffb640 1 -- 172.21.15.122:0/2791739703 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f3ec40b6210 con 0x7f3ec813f4a0 2024-03-16T20:46:02.042 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.038+0000 7f3ecf60c640 1 -- 172.21.15.122:0/2791739703 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3ec80b76e0 con 0x7f3ec813f4a0 2024-03-16T20:46:02.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.698+0000 7f3ebdffb640 1 -- 172.21.15.122:0/2791739703 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3ec40180a0 con 0x7f3ec813f4a0 2024-03-16T20:46:02.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.702+0000 7f3ecf60c640 1 -- 172.21.15.122:0/2791739703 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3eac066e40 msgr2=0x7f3eac069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:02.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.702+0000 7f3ecf60c640 1 --2- 172.21.15.122:0/2791739703 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3eac066e40 0x7f3eac069300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3eb80ad000 tx=0x7f3eb8004740 comp rx=0 tx=0).stop 2024-03-16T20:46:02.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.702+0000 7f3ecf60c640 1 -- 172.21.15.122:0/2791739703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ec813f4a0 msgr2=0x7f3ec80af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:02.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.702+0000 7f3ecf60c640 1 --2- 172.21.15.122:0/2791739703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ec813f4a0 0x7f3ec80af440 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400c450 tx=0x7f3ec400c910 comp rx=0 tx=0).stop 2024-03-16T20:46:02.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.702+0000 7f3ecf60c640 1 -- 172.21.15.122:0/2791739703 shutdown_connections 2024-03-16T20:46:02.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.702+0000 7f3ecf60c640 1 --2- 172.21.15.122:0/2791739703 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3eac066e40 0x7f3eac069300 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:02.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.702+0000 7f3ecf60c640 1 --2- 172.21.15.122:0/2791739703 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3ec8145510 0x7f3ec8132080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:02.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.702+0000 7f3ecf60c640 1 --2- 172.21.15.122:0/2791739703 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3ec813f4a0 0x7f3ec80af440 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:02.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.702+0000 7f3ecf60c640 1 --2- 172.21.15.122:0/2791739703 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3ec806e030 0x7f3ec8139010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:02.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.702+0000 7f3ecf60c640 1 -- 172.21.15.122:0/2791739703 >> 172.21.15.122:0/2791739703 conn(0x7f3ec806d2b0 msgr2=0x7f3ec80b4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:02.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.702+0000 7f3ecf60c640 1 -- 172.21.15.122:0/2791739703 shutdown_connections 2024-03-16T20:46:02.709 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.702+0000 7f3ecf60c640 1 -- 172.21.15.122:0/2791739703 wait complete. 2024-03-16T20:46:02.728 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-16T20:46:02.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9e07c6640 1 Processor -- start 2024-03-16T20:46:02.896 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9e07c6640 1 -- start start 2024-03-16T20:46:02.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9e07c6640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fc9d809ae30 0x7fc9d80af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:02.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9e07c6640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc9d80b4020 0x7fc9d80b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:02.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9e07c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9d80b7820 0x7fc9d80b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:02.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9e07c6640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d80b38e0 con 0x7fc9d80b4020 2024-03-16T20:46:02.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9e07c6640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d809b210 con 0x7fc9d809ae30 2024-03-16T20:46:02.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9e07c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80b4fb0 con 0x7fc9d80b7820 2024-03-16T20:46:02.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9ddd61640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc9d80b4020 0x7fc9d80b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:02.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9ddd61640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc9d80b4020 0x7fc9d80b67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40756/0 (socket says 172.21.15.122:40756) 2024-03-16T20:46:02.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9ddd61640 1 -- 172.21.15.122:0/3465069633 learned_addr learned my addr 172.21.15.122:0/3465069633 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:02.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9de562640 1 --2- 172.21.15.122:0/3465069633 >> v2:172.21.15.195:3300/0 conn(0x7fc9d809ae30 0x7fc9d80af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:02.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9ded63640 1 --2- 172.21.15.122:0/3465069633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9d80b7820 0x7fc9d80b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:02.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9ddd61640 1 -- 172.21.15.122:0/3465069633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9d80b7820 msgr2=0x7fc9d80b6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:02.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9ddd61640 1 --2- 172.21.15.122:0/3465069633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9d80b7820 0x7fc9d80b6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:02.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9ddd61640 1 -- 172.21.15.122:0/3465069633 >> v2:172.21.15.195:3300/0 conn(0x7fc9d809ae30 msgr2=0x7fc9d80af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:02.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9ddd61640 1 --2- 172.21.15.122:0/3465069633 >> v2:172.21.15.195:3300/0 conn(0x7fc9d809ae30 0x7fc9d80af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:02.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9ddd61640 1 -- 172.21.15.122:0/3465069633 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d80b3290 con 0x7fc9d80b4020 2024-03-16T20:46:02.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.890+0000 7fc9ded63640 1 --2- 172.21.15.122:0/3465069633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9d80b7820 0x7fc9d80b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:02.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.894+0000 7fc9ddd61640 1 --2- 172.21.15.122:0/3465069633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9d80b4020 0x7fc9d80b67f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80b9780 tx=0x7fc9c801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4bbeca448a9d1 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:02.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.894+0000 7fc9dd560640 1 -- 172.21.15.122:0/3465069633 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80ab670 con 0x7fc9d80b4020 2024-03-16T20:46:02.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.894+0000 7fc9dd560640 1 -- 172.21.15.122:0/3465069633 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c80ab7d0 con 0x7fc9d80b4020 2024-03-16T20:46:02.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.894+0000 7fc9dd560640 1 -- 172.21.15.122:0/3465069633 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80abb60 con 0x7fc9d80b4020 2024-03-16T20:46:02.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.894+0000 7fc9e07c6640 1 -- 172.21.15.122:0/3465069633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9d80b4020 msgr2=0x7fc9d80b67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:02.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.894+0000 7fc9e07c6640 1 --2- 172.21.15.122:0/3465069633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9d80b4020 0x7fc9d80b67f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc9d80b9780 tx=0x7fc9c801c490 comp rx=0 tx=0).stop 2024-03-16T20:46:02.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.894+0000 7fc9e07c6640 1 -- 172.21.15.122:0/3465069633 shutdown_connections 2024-03-16T20:46:02.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.894+0000 7fc9e07c6640 1 --2- 172.21.15.122:0/3465069633 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9d80b7820 0x7fc9d80b6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:02.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.894+0000 7fc9e07c6640 1 --2- 172.21.15.122:0/3465069633 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9d80b4020 0x7fc9d80b67f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:02.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.894+0000 7fc9e07c6640 1 --2- 172.21.15.122:0/3465069633 >> v2:172.21.15.195:3300/0 conn(0x7fc9d809ae30 0x7fc9d80af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:02.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.894+0000 7fc9e07c6640 1 -- 172.21.15.122:0/3465069633 >> 172.21.15.122:0/3465069633 conn(0x7fc9d80b8d10 msgr2=0x7fc9d809a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:02.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.894+0000 7fc9e07c6640 1 -- 172.21.15.122:0/3465069633 shutdown_connections 2024-03-16T20:46:02.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.894+0000 7fc9e07c6640 1 -- 172.21.15.122:0/3465069633 wait complete. 2024-03-16T20:46:02.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.894+0000 7fc9e07c6640 1 Processor -- start 2024-03-16T20:46:02.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.894+0000 7fc9e07c6640 1 -- start start 2024-03-16T20:46:02.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9e07c6640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9d809ae30 0x7fc9d809cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:02.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9e07c6640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d80b4020 0x7fc9d814cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:02.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9e07c6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9d80b7820 0x7fc9d814d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:02.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9e07c6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d80b09f0 con 0x7fc9d80b7820 2024-03-16T20:46:02.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9e07c6640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d814e230 con 0x7fc9d80b4020 2024-03-16T20:46:02.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9e07c6640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d809d4d0 con 0x7fc9d809ae30 2024-03-16T20:46:02.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9ded63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9d80b7820 0x7fc9d814d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:02.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9ded63640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9d80b7820 0x7fc9d814d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40772/0 (socket says 172.21.15.122:40772) 2024-03-16T20:46:02.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9ded63640 1 -- 172.21.15.122:0/390741706 learned_addr learned my addr 172.21.15.122:0/390741706 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:02.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9de562640 1 --2- 172.21.15.122:0/390741706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9d809ae30 0x7fc9d809cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:02.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9ded63640 1 -- 172.21.15.122:0/390741706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9d809ae30 msgr2=0x7fc9d809cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:02.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9ded63640 1 --2- 172.21.15.122:0/390741706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9d809ae30 0x7fc9d809cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:02.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9ded63640 1 -- 172.21.15.122:0/390741706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d80b4020 msgr2=0x7fc9d814cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:02.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9ddd61640 1 --2- 172.21.15.122:0/390741706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d80b4020 0x7fc9d814cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:02.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9ded63640 1 --2- 172.21.15.122:0/390741706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d80b4020 0x7fc9d814cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:02.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9ded63640 1 -- 172.21.15.122:0/390741706 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c80094d0 con 0x7fc9d80b7820 2024-03-16T20:46:02.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9ded63640 1 --2- 172.21.15.122:0/390741706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9d80b7820 0x7fc9d814d330 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000cd50 tx=0x7fc9d000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:02.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9ceffd640 1 -- 172.21.15.122:0/390741706 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d000f680 con 0x7fc9d80b7820 2024-03-16T20:46:02.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9ceffd640 1 -- 172.21.15.122:0/390741706 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d000f800 con 0x7fc9d80b7820 2024-03-16T20:46:02.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9e07c6640 1 -- 172.21.15.122:0/390741706 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d814c8d0 con 0x7fc9d80b7820 2024-03-16T20:46:02.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9ceffd640 1 -- 172.21.15.122:0/390741706 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d000fbb0 con 0x7fc9d80b7820 2024-03-16T20:46:02.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9e07c6640 1 -- 172.21.15.122:0/390741706 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d809b5b0 con 0x7fc9d80b7820 2024-03-16T20:46:02.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9e07c6640 1 -- 172.21.15.122:0/390741706 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d814cb10 con 0x7fc9d80b7820 2024-03-16T20:46:02.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9ceffd640 1 -- 172.21.15.122:0/390741706 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fc9d000b040 con 0x7fc9d80b7820 2024-03-16T20:46:02.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9ceffd640 1 --2- 172.21.15.122:0/390741706 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc9bc066e70 0x7fc9bc069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:02.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9ceffd640 1 -- 172.21.15.122:0/390741706 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc9d0085320 con 0x7fc9d80b7820 2024-03-16T20:46:02.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.898+0000 7fc9de562640 1 --2- 172.21.15.122:0/390741706 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc9bc066e70 0x7fc9bc069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:02.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.902+0000 7fc9de562640 1 --2- 172.21.15.122:0/390741706 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc9bc066e70 0x7fc9bc069330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009790 tx=0x7fc9d4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:02.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:02.902+0000 7fc9ceffd640 1 -- 172.21.15.122:0/390741706 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc9d814cb10 con 0x7fc9d80b7820 2024-03-16T20:46:03.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:03.066+0000 7fc9e07c6640 1 -- 172.21.15.122:0/390741706 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc9d814cb10 con 0x7fc9d80b7820 2024-03-16T20:46:04.031 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.026+0000 7fc9ceffd640 1 -- 172.21.15.122:0/390741706 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v32) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc9d814cb10 con 0x7fc9d80b7820 2024-03-16T20:46:04.032 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_metadata' removed 2024-03-16T20:46:04.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.030+0000 7fc9e07c6640 1 -- 172.21.15.122:0/390741706 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc9bc066e70 msgr2=0x7fc9bc069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:04.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.030+0000 7fc9e07c6640 1 --2- 172.21.15.122:0/390741706 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc9bc066e70 0x7fc9bc069330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc9d4009790 tx=0x7fc9d4008040 comp rx=0 tx=0).stop 2024-03-16T20:46:04.035 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.030+0000 7fc9e07c6640 1 -- 172.21.15.122:0/390741706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9d80b7820 msgr2=0x7fc9d814d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:04.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.030+0000 7fc9e07c6640 1 --2- 172.21.15.122:0/390741706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9d80b7820 0x7fc9d814d330 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000cd50 tx=0x7fc9d000a650 comp rx=0 tx=0).stop 2024-03-16T20:46:04.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.030+0000 7fc9e07c6640 1 -- 172.21.15.122:0/390741706 shutdown_connections 2024-03-16T20:46:04.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.030+0000 7fc9e07c6640 1 --2- 172.21.15.122:0/390741706 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc9bc066e70 0x7fc9bc069330 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:04.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.030+0000 7fc9e07c6640 1 --2- 172.21.15.122:0/390741706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc9d80b7820 0x7fc9d814d330 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:04.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.030+0000 7fc9e07c6640 1 --2- 172.21.15.122:0/390741706 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc9d80b4020 0x7fc9d814cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:04.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.030+0000 7fc9e07c6640 1 --2- 172.21.15.122:0/390741706 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc9d809ae30 0x7fc9d809cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:04.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.030+0000 7fc9e07c6640 1 -- 172.21.15.122:0/390741706 >> 172.21.15.122:0/390741706 conn(0x7fc9d80b8d10 msgr2=0x7fc9d8145a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:04.036 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.030+0000 7fc9e07c6640 1 -- 172.21.15.122:0/390741706 shutdown_connections 2024-03-16T20:46:04.037 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.030+0000 7fc9e07c6640 1 -- 172.21.15.122:0/390741706 wait complete. 2024-03-16T20:46:04.057 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-16T20:46:04.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.174+0000 7fd6d6c42640 1 Processor -- start 2024-03-16T20:46:04.179 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.174+0000 7fd6d6c42640 1 -- start start 2024-03-16T20:46:04.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.174+0000 7fd6d6c42640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd6d0144aa0 0x7fd6d014f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:04.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.174+0000 7fd6d6c42640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d00b4270 con 0x7fd6d009ae30 2024-03-16T20:46:04.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.174+0000 7fd6d6c42640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d009b210 con 0x7fd6d00aef80 2024-03-16T20:46:04.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.174+0000 7fd6d6c42640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d00b3110 con 0x7fd6d0144aa0 2024-03-16T20:46:04.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.174+0000 7fd6d49de640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fd6d009ae30 0x7fd6d00b3b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:40570/0 (socket says 172.21.15.122:40570) 2024-03-16T20:46:04.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.174+0000 7fd6d49de640 1 -- 172.21.15.122:0/1820766523 learned_addr learned my addr 172.21.15.122:0/1820766523 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:46:04.180 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.174+0000 7fd6d51df640 1 --2- 172.21.15.122:0/1820766523 >> v2:172.21.15.122:3300/0 conn(0x7fd6d0144aa0 0x7fd6d014f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:04.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.174+0000 7fd6cf7fe640 1 -- 172.21.15.122:0/1820766523 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513239247 0 0) 0x7fd6d00b4270 con 0x7fd6d009ae30 2024-03-16T20:46:04.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.174+0000 7fd6cf7fe640 1 -- 172.21.15.122:0/1820766523 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6b8003530 con 0x7fd6d009ae30 2024-03-16T20:46:04.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.174+0000 7fd6d51df640 1 -- 172.21.15.122:0/1820766523 >> v1:172.21.15.195:6789/0 conn(0x7fd6d00aef80 legacy=0x7fd6d0144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:04.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.174+0000 7fd6d51df640 1 -- 172.21.15.122:0/1820766523 >> v1:172.21.15.122:6789/0 conn(0x7fd6d009ae30 legacy=0x7fd6d00b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:04.181 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.174+0000 7fd6d51df640 1 -- 172.21.15.122:0/1820766523 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d00b3320 con 0x7fd6d0144aa0 2024-03-16T20:46:04.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d51df640 1 --2- 172.21.15.122:0/1820766523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6d0144aa0 0x7fd6d014f860 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009d90 tx=0x7fd6c801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5333716fdfec7651 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:04.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6cf7fe640 1 -- 172.21.15.122:0/1820766523 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c80a57a0 con 0x7fd6d0144aa0 2024-03-16T20:46:04.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6cf7fe640 1 -- 172.21.15.122:0/1820766523 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c80a5900 con 0x7fd6d0144aa0 2024-03-16T20:46:04.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6cf7fe640 1 -- 172.21.15.122:0/1820766523 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c80a5c90 con 0x7fd6d0144aa0 2024-03-16T20:46:04.183 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d6c42640 1 -- 172.21.15.122:0/1820766523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6d0144aa0 msgr2=0x7fd6d014f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:04.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d6c42640 1 --2- 172.21.15.122:0/1820766523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6d0144aa0 0x7fd6d014f860 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009d90 tx=0x7fd6c801c8e0 comp rx=0 tx=0).stop 2024-03-16T20:46:04.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d6c42640 1 -- 172.21.15.122:0/1820766523 shutdown_connections 2024-03-16T20:46:04.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d6c42640 1 --2- 172.21.15.122:0/1820766523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6d0144aa0 0x7fd6d014f860 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:04.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d6c42640 1 -- 172.21.15.122:0/1820766523 >> 172.21.15.122:0/1820766523 conn(0x7fd6d00b8d10 msgr2=0x7fd6d009a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:04.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d6c42640 1 -- 172.21.15.122:0/1820766523 shutdown_connections 2024-03-16T20:46:04.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d6c42640 1 -- 172.21.15.122:0/1820766523 wait complete. 2024-03-16T20:46:04.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d6c42640 1 Processor -- start 2024-03-16T20:46:04.184 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d6c42640 1 -- start start 2024-03-16T20:46:04.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d6c42640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6d009ae30 0x7fd6d0071900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:04.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d6c42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6d00aef80 0x7fd6d00b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:04.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d6c42640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6d0144aa0 0x7fd6d01e2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:04.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d6c42640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d00b0fe0 con 0x7fd6d00aef80 2024-03-16T20:46:04.186 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d6c42640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d00b25d0 con 0x7fd6d009ae30 2024-03-16T20:46:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d6c42640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d00b2b50 con 0x7fd6d0144aa0 2024-03-16T20:46:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6d00aef80 0x7fd6d00b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6cffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6d00aef80 0x7fd6d00b1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40782/0 (socket says 172.21.15.122:40782) 2024-03-16T20:46:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6cffff640 1 -- 172.21.15.122:0/3741205864 learned_addr learned my addr 172.21.15.122:0/3741205864 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d49de640 1 --2- 172.21.15.122:0/3741205864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6d009ae30 0x7fd6d0071900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6cffff640 1 -- 172.21.15.122:0/3741205864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6d0144aa0 msgr2=0x7fd6d01e2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6cffff640 1 --2- 172.21.15.122:0/3741205864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6d0144aa0 0x7fd6d01e2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6cffff640 1 -- 172.21.15.122:0/3741205864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6d009ae30 msgr2=0x7fd6d0071900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6cffff640 1 --2- 172.21.15.122:0/3741205864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6d009ae30 0x7fd6d0071900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6cffff640 1 -- 172.21.15.122:0/3741205864 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c8009a70 con 0x7fd6d00aef80 2024-03-16T20:46:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6cffff640 1 --2- 172.21.15.122:0/3741205864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6d00aef80 0x7fd6d00b1470 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400c420 tx=0x7fd6c400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6cd7fa640 1 -- 172.21.15.122:0/3741205864 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c40052d0 con 0x7fd6d00aef80 2024-03-16T20:46:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6cd7fa640 1 -- 172.21.15.122:0/3741205864 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c4005430 con 0x7fd6d00aef80 2024-03-16T20:46:04.187 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6d6c42640 1 -- 172.21.15.122:0/3741205864 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d014a410 con 0x7fd6d00aef80 2024-03-16T20:46:04.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.178+0000 7fd6cd7fa640 1 -- 172.21.15.122:0/3741205864 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4007260 con 0x7fd6d00aef80 2024-03-16T20:46:04.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.182+0000 7fd6d6c42640 1 -- 172.21.15.122:0/3741205864 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d00b2870 con 0x7fd6d00aef80 2024-03-16T20:46:04.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.182+0000 7fd6d6c42640 1 -- 172.21.15.122:0/3741205864 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d00b1150 con 0x7fd6d00aef80 2024-03-16T20:46:04.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.182+0000 7fd6cd7fa640 1 -- 172.21.15.122:0/3741205864 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fd6c40073c0 con 0x7fd6d00aef80 2024-03-16T20:46:04.188 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.182+0000 7fd6cd7fa640 1 --2- 172.21.15.122:0/3741205864 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd6bc066e40 0x7fd6bc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.182+0000 7fd6cd7fa640 1 -- 172.21.15.122:0/3741205864 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fd6c4083c70 con 0x7fd6d00aef80 2024-03-16T20:46:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.182+0000 7fd6d49de640 1 --2- 172.21.15.122:0/3741205864 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd6bc066e40 0x7fd6bc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:04.189 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.182+0000 7fd6d49de640 1 --2- 172.21.15.122:0/3741205864 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd6bc066e40 0x7fd6bc069300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0005890 tx=0x7fd6c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:04.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.186+0000 7fd6cd7fa640 1 -- 172.21.15.122:0/3741205864 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd6c4050c50 con 0x7fd6d00aef80 2024-03-16T20:46:04.350 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:04.342+0000 7fd6d6c42640 1 -- 172.21.15.122:0/3741205864 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd6d0143cb0 con 0x7fd6d00aef80 2024-03-16T20:46:05.043 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.038+0000 7fd6cd7fa640 1 -- 172.21.15.122:0/3741205864 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v33) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd6c4054310 con 0x7fd6d00aef80 2024-03-16T20:46:05.043 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'cephfs_data' removed 2024-03-16T20:46:05.046 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.042+0000 7fd6d6c42640 1 -- 172.21.15.122:0/3741205864 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd6bc066e40 msgr2=0x7fd6bc069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:05.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.042+0000 7fd6d6c42640 1 --2- 172.21.15.122:0/3741205864 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd6bc066e40 0x7fd6bc069300 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0005890 tx=0x7fd6c0009040 comp rx=0 tx=0).stop 2024-03-16T20:46:05.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.042+0000 7fd6d6c42640 1 -- 172.21.15.122:0/3741205864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6d00aef80 msgr2=0x7fd6d00b1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:05.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.042+0000 7fd6d6c42640 1 --2- 172.21.15.122:0/3741205864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6d00aef80 0x7fd6d00b1470 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400c420 tx=0x7fd6c400c8e0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.042+0000 7fd6d6c42640 1 -- 172.21.15.122:0/3741205864 shutdown_connections 2024-03-16T20:46:05.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.042+0000 7fd6d6c42640 1 --2- 172.21.15.122:0/3741205864 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd6bc066e40 0x7fd6bc069300 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.047 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.042+0000 7fd6d6c42640 1 --2- 172.21.15.122:0/3741205864 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd6d0144aa0 0x7fd6d01e2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.042+0000 7fd6d6c42640 1 --2- 172.21.15.122:0/3741205864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd6d00aef80 0x7fd6d00b1470 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.042+0000 7fd6d6c42640 1 --2- 172.21.15.122:0/3741205864 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd6d009ae30 0x7fd6d0071900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.042+0000 7fd6d6c42640 1 -- 172.21.15.122:0/3741205864 >> 172.21.15.122:0/3741205864 conn(0x7fd6d00b8d10 msgr2=0x7fd6d0142b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:05.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.042+0000 7fd6d6c42640 1 -- 172.21.15.122:0/3741205864 shutdown_connections 2024-03-16T20:46:05.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.042+0000 7fd6d6c42640 1 -- 172.21.15.122:0/3741205864 wait complete. 2024-03-16T20:46:05.072 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-03-16T20:46:05.192 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.186+0000 7f59603c0640 1 Processor -- start 2024-03-16T20:46:05.193 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.186+0000 7f59603c0640 1 -- start start 2024-03-16T20:46:05.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.186+0000 7f59603c0640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f59580b6f00 0x7f595814f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:05.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.186+0000 7f59603c0640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59580b38e0 con 0x7f59580b4020 2024-03-16T20:46:05.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.186+0000 7f59603c0640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f595809b210 con 0x7f595809ae30 2024-03-16T20:46:05.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.186+0000 7f59603c0640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59580af790 con 0x7f59580b6f00 2024-03-16T20:46:05.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.186+0000 7f595e95d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f59580b6f00 0x7f595814f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:05.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.186+0000 7f595e95d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f59580b6f00 0x7f595814f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40784/0 (socket says 172.21.15.122:40784) 2024-03-16T20:46:05.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.186+0000 7f595e95d640 1 -- 172.21.15.122:0/2930662597 learned_addr learned my addr 172.21.15.122:0/2930662597 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:05.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.186+0000 7f595e95d640 1 -- 172.21.15.122:0/2930662597 >> v1:172.21.15.195:6789/0 conn(0x7f595809ae30 legacy=0x7f59580af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:05.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.186+0000 7f595e95d640 1 -- 172.21.15.122:0/2930662597 >> v1:172.21.15.122:6789/0 conn(0x7f59580b4020 legacy=0x7f59580b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:05.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.186+0000 7f595d15a640 1 -- 172.21.15.122:0/2930662597 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625979332 0 0) 0x7f59580b38e0 con 0x7f59580b4020 2024-03-16T20:46:05.194 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.186+0000 7f595e95d640 1 -- 172.21.15.122:0/2930662597 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59580b4fb0 con 0x7f59580b6f00 2024-03-16T20:46:05.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.190+0000 7f595e95d640 1 --2- 172.21.15.122:0/2930662597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59580b6f00 0x7f595814f750 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5950009e50 tx=0x7f595001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f24964c99bb396 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:05.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.190+0000 7f595d15a640 1 -- 172.21.15.122:0/2930662597 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500a5830 con 0x7f59580b6f00 2024-03-16T20:46:05.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.190+0000 7f595d15a640 1 -- 172.21.15.122:0/2930662597 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59500a5990 con 0x7f59580b6f00 2024-03-16T20:46:05.196 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.190+0000 7f595d15a640 1 -- 172.21.15.122:0/2930662597 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500a5d20 con 0x7f59580b6f00 2024-03-16T20:46:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.190+0000 7f59603c0640 1 -- 172.21.15.122:0/2930662597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59580b6f00 msgr2=0x7f595814f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.190+0000 7f59603c0640 1 --2- 172.21.15.122:0/2930662597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59580b6f00 0x7f595814f750 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5950009e50 tx=0x7f595001c970 comp rx=0 tx=0).stop 2024-03-16T20:46:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.190+0000 7f59603c0640 1 -- 172.21.15.122:0/2930662597 shutdown_connections 2024-03-16T20:46:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.190+0000 7f59603c0640 1 --2- 172.21.15.122:0/2930662597 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59580b6f00 0x7f595814f750 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.190+0000 7f59603c0640 1 -- 172.21.15.122:0/2930662597 >> 172.21.15.122:0/2930662597 conn(0x7f59580b8d10 msgr2=0x7f595809a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.190+0000 7f59603c0640 1 -- 172.21.15.122:0/2930662597 shutdown_connections 2024-03-16T20:46:05.197 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.190+0000 7f59603c0640 1 -- 172.21.15.122:0/2930662597 wait complete. 2024-03-16T20:46:05.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.190+0000 7f59603c0640 1 Processor -- start 2024-03-16T20:46:05.198 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f59603c0640 1 -- start start 2024-03-16T20:46:05.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f59603c0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f595809ae30 0x7f595813cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:05.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f59603c0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59580b4020 0x7f59580b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:05.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f59603c0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59580b6f00 0x7f59580b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:05.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f59603c0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59581480c0 con 0x7f59580b6f00 2024-03-16T20:46:05.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f59603c0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59580b13b0 con 0x7f59580b4020 2024-03-16T20:46:05.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f59603c0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5958147eb0 con 0x7f595809ae30 2024-03-16T20:46:05.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f595e95d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59580b6f00 0x7f59580b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:05.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f595e95d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59580b6f00 0x7f59580b0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40792/0 (socket says 172.21.15.122:40792) 2024-03-16T20:46:05.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f595e95d640 1 -- 172.21.15.122:0/1683313553 learned_addr learned my addr 172.21.15.122:0/1683313553 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:05.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f595e95d640 1 -- 172.21.15.122:0/1683313553 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f595809ae30 msgr2=0x7f595813cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:05.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f595e95d640 1 --2- 172.21.15.122:0/1683313553 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f595809ae30 0x7f595813cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.200 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f595e95d640 1 -- 172.21.15.122:0/1683313553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59580b4020 msgr2=0x7f59580b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:05.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f595e95d640 1 --2- 172.21.15.122:0/1683313553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59580b4020 0x7f59580b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f595e95d640 1 -- 172.21.15.122:0/1683313553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5950009b30 con 0x7f59580b6f00 2024-03-16T20:46:05.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f595e95d640 1 --2- 172.21.15.122:0/1683313553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59580b6f00 0x7f59580b0a30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5950019580 tx=0x7f5950019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:05.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f594effd640 1 -- 172.21.15.122:0/1683313553 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500b8670 con 0x7f59580b6f00 2024-03-16T20:46:05.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f594effd640 1 -- 172.21.15.122:0/1683313553 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59500b87f0 con 0x7f59580b6f00 2024-03-16T20:46:05.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f59603c0640 1 -- 172.21.15.122:0/1683313553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59580b2310 con 0x7f59580b6f00 2024-03-16T20:46:05.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f594effd640 1 -- 172.21.15.122:0/1683313553 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500b8ba0 con 0x7f59580b6f00 2024-03-16T20:46:05.201 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f59603c0640 1 -- 172.21.15.122:0/1683313553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59580aff90 con 0x7f59580b6f00 2024-03-16T20:46:05.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f59603c0640 1 -- 172.21.15.122:0/1683313553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59580b0180 con 0x7f59580b6f00 2024-03-16T20:46:05.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f594effd640 1 -- 172.21.15.122:0/1683313553 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f59500b8d00 con 0x7f59580b6f00 2024-03-16T20:46:05.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f594effd640 1 --2- 172.21.15.122:0/1683313553 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f593c066e40 0x7f593c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:05.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f594effd640 1 -- 172.21.15.122:0/1683313553 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f595011fef0 con 0x7f59580b6f00 2024-03-16T20:46:05.202 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.194+0000 7f595e15c640 1 --2- 172.21.15.122:0/1683313553 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f593c066e40 0x7f593c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:05.203 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.198+0000 7f595e15c640 1 --2- 172.21.15.122:0/1683313553 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f593c066e40 0x7f593c069300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5948000f70 tx=0x7f59480079c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:05.206 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.198+0000 7f594effd640 1 -- 172.21.15.122:0/1683313553 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f59500ed110 con 0x7f59580b6f00 2024-03-16T20:46:05.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.358+0000 7f59603c0640 1 -- 172.21.15.122:0/1683313553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5958074bb0 con 0x7f59580b6f00 2024-03-16T20:46:05.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.366+0000 7f594effd640 1 -- 172.21.15.122:0/1683313553 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f59500f07d0 con 0x7f59580b6f00 2024-03-16T20:46:05.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.370+0000 7f59603c0640 1 -- 172.21.15.122:0/1683313553 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f593c066e40 msgr2=0x7f593c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:05.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.370+0000 7f59603c0640 1 --2- 172.21.15.122:0/1683313553 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f593c066e40 0x7f593c069300 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5948000f70 tx=0x7f59480079c0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.370+0000 7f59603c0640 1 -- 172.21.15.122:0/1683313553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59580b6f00 msgr2=0x7f59580b0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:05.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.370+0000 7f59603c0640 1 --2- 172.21.15.122:0/1683313553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59580b6f00 0x7f59580b0a30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f5950019580 tx=0x7f5950019710 comp rx=0 tx=0).stop 2024-03-16T20:46:05.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.370+0000 7f59603c0640 1 -- 172.21.15.122:0/1683313553 shutdown_connections 2024-03-16T20:46:05.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.370+0000 7f59603c0640 1 --2- 172.21.15.122:0/1683313553 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f593c066e40 0x7f593c069300 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.370+0000 7f59603c0640 1 --2- 172.21.15.122:0/1683313553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f59580b6f00 0x7f59580b0a30 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.370+0000 7f59603c0640 1 --2- 172.21.15.122:0/1683313553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f59580b4020 0x7f59580b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.370+0000 7f59603c0640 1 --2- 172.21.15.122:0/1683313553 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f595809ae30 0x7f595813cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.370+0000 7f59603c0640 1 -- 172.21.15.122:0/1683313553 >> 172.21.15.122:0/1683313553 conn(0x7f59580b8d10 msgr2=0x7f59580b5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:05.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.374+0000 7f59603c0640 1 -- 172.21.15.122:0/1683313553 shutdown_connections 2024-03-16T20:46:05.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.374+0000 7f59603c0640 1 -- 172.21.15.122:0/1683313553 wait complete. 2024-03-16T20:46:05.397 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args' 2024-03-16T20:46:05.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.514+0000 7f59477ed640 1 Processor -- start 2024-03-16T20:46:05.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.514+0000 7f59477ed640 1 -- start start 2024-03-16T20:46:05.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.514+0000 7f59477ed640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f594013ed60 0x7f594013f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:05.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.514+0000 7f59477ed640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f5940148430 0x7f5940148880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:05.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.514+0000 7f59477ed640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59400b02a0 con 0x7f5940072120 2024-03-16T20:46:05.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.514+0000 7f59477ed640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5940148dc0 con 0x7f594013ed60 2024-03-16T20:46:05.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.514+0000 7f59477ed640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59400b1050 con 0x7f5940148430 2024-03-16T20:46:05.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.514+0000 7f5944d88640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f594013ed60 0x7f594013f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:05.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.514+0000 7f5944d88640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f594013ed60 0x7f594013f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40804/0 (socket says 172.21.15.122:40804) 2024-03-16T20:46:05.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.514+0000 7f5944d88640 1 -- 172.21.15.122:0/95172896 learned_addr learned my addr 172.21.15.122:0/95172896 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:05.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.514+0000 7f5945d8a640 1 --2- 172.21.15.122:0/95172896 >> v2:172.21.15.195:3300/0 conn(0x7f5940148430 0x7f5940148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:05.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.514+0000 7f5944d88640 1 -- 172.21.15.122:0/95172896 >> v2:172.21.15.195:3300/0 conn(0x7f5940148430 msgr2=0x7f5940148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:05.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.514+0000 7f5944d88640 1 --2- 172.21.15.122:0/95172896 >> v2:172.21.15.195:3300/0 conn(0x7f5940148430 0x7f5940148880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.514+0000 7f5944d88640 1 -- 172.21.15.122:0/95172896 >> v1:172.21.15.195:6789/0 conn(0x7f5940072120 legacy=0x7f59400afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:05.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.514+0000 7f5944d88640 1 -- 172.21.15.122:0/95172896 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5940072500 con 0x7f594013ed60 2024-03-16T20:46:05.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.514+0000 7f5945d8a640 1 --2- 172.21.15.122:0/95172896 >> v2:172.21.15.195:3300/0 conn(0x7f5940148430 0x7f5940148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:05.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f5944d88640 1 --2- 172.21.15.122:0/95172896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f594013ed60 0x7f594013f1c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5940148f00 tx=0x7f593001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d997bd878bdc8f5a server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:05.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f5937fff640 1 -- 172.21.15.122:0/95172896 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59300ab600 con 0x7f594013ed60 2024-03-16T20:46:05.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f5937fff640 1 -- 172.21.15.122:0/95172896 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59300a7050 con 0x7f594013ed60 2024-03-16T20:46:05.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f5937fff640 1 -- 172.21.15.122:0/95172896 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59300ab980 con 0x7f594013ed60 2024-03-16T20:46:05.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f59477ed640 1 -- 172.21.15.122:0/95172896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f594013ed60 msgr2=0x7f594013f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:05.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f59477ed640 1 --2- 172.21.15.122:0/95172896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f594013ed60 0x7f594013f1c0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f5940148f00 tx=0x7f593001c450 comp rx=0 tx=0).stop 2024-03-16T20:46:05.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f59477ed640 1 -- 172.21.15.122:0/95172896 shutdown_connections 2024-03-16T20:46:05.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f59477ed640 1 --2- 172.21.15.122:0/95172896 >> v2:172.21.15.195:3300/0 conn(0x7f5940148430 0x7f5940148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f59477ed640 1 --2- 172.21.15.122:0/95172896 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f594013ed60 0x7f594013f1c0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f59477ed640 1 -- 172.21.15.122:0/95172896 >> 172.21.15.122:0/95172896 conn(0x7f59400b8860 msgr2=0x7f59400b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:05.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f59477ed640 1 -- 172.21.15.122:0/95172896 shutdown_connections 2024-03-16T20:46:05.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f59477ed640 1 -- 172.21.15.122:0/95172896 wait complete. 2024-03-16T20:46:05.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f59477ed640 1 Processor -- start 2024-03-16T20:46:05.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f59477ed640 1 -- start start 2024-03-16T20:46:05.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f59477ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5940072120 0x7f59400717c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:05.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f59477ed640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f594013ed60 0x7f594013ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:05.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f59477ed640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5940148430 0x7f59401df780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:05.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f59477ed640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594014f0a0 con 0x7f5940072120 2024-03-16T20:46:05.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f59477ed640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59401df2f0 con 0x7f5940148430 2024-03-16T20:46:05.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f59477ed640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5940140490 con 0x7f594013ed60 2024-03-16T20:46:05.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f5945589640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5940072120 0x7f59400717c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:05.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f5945589640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5940072120 0x7f59400717c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40816/0 (socket says 172.21.15.122:40816) 2024-03-16T20:46:05.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f5945589640 1 -- 172.21.15.122:0/408582759 learned_addr learned my addr 172.21.15.122:0/408582759 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:05.528 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f5945d8a640 1 --2- 172.21.15.122:0/408582759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5940148430 0x7f59401df780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:05.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f5945589640 1 -- 172.21.15.122:0/408582759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f594013ed60 msgr2=0x7f594013ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:05.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.518+0000 7f5945589640 1 --2- 172.21.15.122:0/408582759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f594013ed60 0x7f594013ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.522+0000 7f5945589640 1 -- 172.21.15.122:0/408582759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5940148430 msgr2=0x7f59401df780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:05.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.522+0000 7f5945589640 1 --2- 172.21.15.122:0/408582759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5940148430 0x7f59401df780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:05.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.522+0000 7f5945589640 1 -- 172.21.15.122:0/408582759 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59300094d0 con 0x7f5940072120 2024-03-16T20:46:05.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.522+0000 7f5945d8a640 1 --2- 172.21.15.122:0/408582759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5940148430 0x7f59401df780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:05.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.522+0000 7f5945589640 1 --2- 172.21.15.122:0/408582759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5940072120 0x7f59400717c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f593c00c480 tx=0x7f593c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:05.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.522+0000 7f5935ffb640 1 -- 172.21.15.122:0/408582759 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c012830 con 0x7f5940072120 2024-03-16T20:46:05.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.522+0000 7f5935ffb640 1 -- 172.21.15.122:0/408582759 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f593c012990 con 0x7f5940072120 2024-03-16T20:46:05.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.522+0000 7f5935ffb640 1 -- 172.21.15.122:0/408582759 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f593c012d40 con 0x7f5940072120 2024-03-16T20:46:05.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.522+0000 7f59477ed640 1 -- 172.21.15.122:0/408582759 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5940140710 con 0x7f5940072120 2024-03-16T20:46:05.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.522+0000 7f59477ed640 1 -- 172.21.15.122:0/408582759 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59400710d0 con 0x7f5940072120 2024-03-16T20:46:05.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.522+0000 7f59477ed640 1 -- 172.21.15.122:0/408582759 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5940075280 con 0x7f5940072120 2024-03-16T20:46:05.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.522+0000 7f5935ffb640 1 -- 172.21.15.122:0/408582759 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f593c01c460 con 0x7f5940072120 2024-03-16T20:46:05.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.522+0000 7f5935ffb640 1 --2- 172.21.15.122:0/408582759 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5924066e70 0x7f5924069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:05.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.522+0000 7f5935ffb640 1 -- 172.21.15.122:0/408582759 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f593c084600 con 0x7f5940072120 2024-03-16T20:46:05.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.522+0000 7f5944d88640 1 --2- 172.21.15.122:0/408582759 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5924066e70 0x7f5924069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:05.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.522+0000 7f5944d88640 1 --2- 172.21.15.122:0/408582759 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5924066e70 0x7f5924069330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5940072790 tx=0x7f5930004740 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:05.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.526+0000 7f5935ffb640 1 -- 172.21.15.122:0/408582759 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f593c051780 con 0x7f5940072120 2024-03-16T20:46:05.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:05.742+0000 7f59477ed640 1 -- 172.21.15.122:0/408582759 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]} v 0) v1 -- 0x7f59400b0d00 con 0x7f5940072120 2024-03-16T20:46:06.045 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.038+0000 7f5935ffb640 1 -- 172.21.15.122:0/408582759 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_cephfs_shell.TestCD.test_cd_with_args"]}]=0 v84) v1 ==== 134+0+0 (secure 0 0 0) 0x7f593c054e40 con 0x7f5940072120 2024-03-16T20:46:06.048 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.042+0000 7f59477ed640 1 -- 172.21.15.122:0/408582759 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5924066e70 msgr2=0x7f5924069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:06.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.042+0000 7f59477ed640 1 --2- 172.21.15.122:0/408582759 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5924066e70 0x7f5924069330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5940072790 tx=0x7f5930004740 comp rx=0 tx=0).stop 2024-03-16T20:46:06.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.042+0000 7f59477ed640 1 -- 172.21.15.122:0/408582759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5940072120 msgr2=0x7f59400717c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:06.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.042+0000 7f59477ed640 1 --2- 172.21.15.122:0/408582759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5940072120 0x7f59400717c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f593c00c480 tx=0x7f593c00c940 comp rx=0 tx=0).stop 2024-03-16T20:46:06.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.042+0000 7f59477ed640 1 -- 172.21.15.122:0/408582759 shutdown_connections 2024-03-16T20:46:06.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.042+0000 7f59477ed640 1 --2- 172.21.15.122:0/408582759 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f5924066e70 0x7f5924069330 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:06.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.042+0000 7f59477ed640 1 --2- 172.21.15.122:0/408582759 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f5940148430 0x7f59401df780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:06.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.042+0000 7f59477ed640 1 --2- 172.21.15.122:0/408582759 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f594013ed60 0x7f594013ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:06.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.042+0000 7f59477ed640 1 --2- 172.21.15.122:0/408582759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5940072120 0x7f59400717c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:06.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.042+0000 7f59477ed640 1 -- 172.21.15.122:0/408582759 >> 172.21.15.122:0/408582759 conn(0x7f59400b8860 msgr2=0x7f59400b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:06.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.042+0000 7f59477ed640 1 -- 172.21.15.122:0/408582759 shutdown_connections 2024-03-16T20:46:06.049 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.042+0000 7f59477ed640 1 -- 172.21.15.122:0/408582759 wait complete. 2024-03-16T20:46:06.069 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed ... ERROR 2024-03-16T20:46:06.069 INFO:tasks.cephfs_test_runner: 2024-03-16T20:46:06.069 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-16T20:46:06.069 INFO:tasks.cephfs_test_runner:ERROR: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-16T20:46:06.069 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa/tasks/cephfs/test_cephfs_shell.py", line 602, in test_cd_with_args 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner: output = self.get_cephfs_shell_script_output(script) 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa/tasks/cephfs/test_cephfs_shell.py", line 161, in get_cephfs_shell_script_output 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner: return ensure_str(self.run_cephfs_shell_script( 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa/tasks/cephfs/test_cephfs_shell.py", line 153, in run_cephfs_shell_script 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner: return mount_x.client_remote.run(args=args, stdout=stdout, 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner: r.wait() 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi122 with status 127: 'cephfs-shell -b /tmp/test-cephfs9i_49ozq' 2024-03-16T20:46:06.070 INFO:tasks.cephfs_test_runner: 2024-03-16T20:46:06.071 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-16T20:46:06.071 INFO:tasks.cephfs_test_runner:Ran 1 test in 40.811s 2024-03-16T20:46:06.071 INFO:tasks.cephfs_test_runner: 2024-03-16T20:46:06.071 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-03-16T20:46:06.071 INFO:tasks.cephfs_test_runner: 2024-03-16T20:46:06.071 INFO:tasks.cephfs_test_runner:====================================================================== 2024-03-16T20:46:06.071 INFO:tasks.cephfs_test_runner:ERROR: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-16T20:46:06.071 INFO:tasks.cephfs_test_runner:Test that when cd is issued with an argument, CWD is changed 2024-03-16T20:46:06.071 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-03-16T20:46:06.071 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-03-16T20:46:06.071 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa/tasks/cephfs/test_cephfs_shell.py", line 602, in test_cd_with_args 2024-03-16T20:46:06.071 INFO:tasks.cephfs_test_runner: output = self.get_cephfs_shell_script_output(script) 2024-03-16T20:46:06.071 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa/tasks/cephfs/test_cephfs_shell.py", line 161, in get_cephfs_shell_script_output 2024-03-16T20:46:06.071 INFO:tasks.cephfs_test_runner: return ensure_str(self.run_cephfs_shell_script( 2024-03-16T20:46:06.071 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa/tasks/cephfs/test_cephfs_shell.py", line 153, in run_cephfs_shell_script 2024-03-16T20:46:06.072 INFO:tasks.cephfs_test_runner: return mount_x.client_remote.run(args=args, stdout=stdout, 2024-03-16T20:46:06.072 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/remote.py", line 523, in run 2024-03-16T20:46:06.072 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2024-03-16T20:46:06.072 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 455, in run 2024-03-16T20:46:06.072 INFO:tasks.cephfs_test_runner: r.wait() 2024-03-16T20:46:06.072 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 161, in wait 2024-03-16T20:46:06.072 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2024-03-16T20:46:06.072 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/orchestra/run.py", line 181, in _raise_for_status 2024-03-16T20:46:06.072 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2024-03-16T20:46:06.072 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi122 with status 127: 'cephfs-shell -b /tmp/test-cephfs9i_49ozq' 2024-03-16T20:46:06.072 INFO:tasks.cephfs_test_runner: 2024-03-16T20:46:06.073 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-16T20:46:06.301 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=37190baf060548af9205195780300456 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_e691533f9cbb33d85b2187bba20d7102f098636d/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_aba3c1d240c873fc0cd3e1d30cd86b2dbb71f24b/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD) 2024-03-16T20:46:06.303 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-03-16T20:46:06.316 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-03-16T20:46:06.337 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-03-16T20:46:06.338 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-03-16T20:46:06.338 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-16T20:46:06.351 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336348k,nr_inodes=4084087,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274660k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15779 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274660k,nr_inodes=818665,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274660k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 2024-03-16T20:46:06.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:46:06.351 DEBUG:teuthology.orchestra.run.smithi122:> ip netns list 2024-03-16T20:46:06.399 INFO:teuthology.orchestra.run.smithi122.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-16T20:46:06.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:46:06.400 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-16T20:46:06.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-16T20:46:06.456 DEBUG:teuthology.orchestra.run.smithi122:> sudo ip link delete ceph-brx 2024-03-16T20:46:06.600 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T20:46:06.602 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T20:46:06.668 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-03-16T20:46:06.677 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-03-16T20:46:06.677 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T20:46:06.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.794+0000 7f4e4c085640 1 Processor -- start 2024-03-16T20:46:06.802 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.794+0000 7f4e4c085640 1 -- start start 2024-03-16T20:46:06.803 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e4c085640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f4e44146a50 0x7f4e44143460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:06.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e4c085640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f4e44071580 0x7f4e441439a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:06.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e4c085640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e440b4c50 con 0x7f4e4413d8d0 2024-03-16T20:46:06.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e4c085640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e440b2c30 con 0x7f4e44071580 2024-03-16T20:46:06.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e4c085640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4406bf30 con 0x7f4e44146a50 2024-03-16T20:46:06.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e49e21640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f4e4413d8d0 0x7f4e4406b820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:40584/0 (socket says 172.21.15.122:40584) 2024-03-16T20:46:06.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e49e21640 1 -- 172.21.15.122:0/1498854579 learned_addr learned my addr 172.21.15.122:0/1498854579 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:46:06.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e4a622640 1 --2- 172.21.15.122:0/1498854579 >> v2:172.21.15.122:3300/0 conn(0x7f4e44071580 0x7f4e441439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:06.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e49620640 1 --2- 172.21.15.122:0/1498854579 >> v2:172.21.15.195:3300/0 conn(0x7f4e44146a50 0x7f4e44143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:06.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e48e1f640 1 -- 172.21.15.122:0/1498854579 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670363439 0 0) 0x7f4e440b4c50 con 0x7f4e4413d8d0 2024-03-16T20:46:06.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e48e1f640 1 -- 172.21.15.122:0/1498854579 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e30003530 con 0x7f4e4413d8d0 2024-03-16T20:46:06.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e4a622640 1 -- 172.21.15.122:0/1498854579 >> v2:172.21.15.195:3300/0 conn(0x7f4e44146a50 msgr2=0x7f4e44143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:06.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e4a622640 1 --2- 172.21.15.122:0/1498854579 >> v2:172.21.15.195:3300/0 conn(0x7f4e44146a50 0x7f4e44143460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:06.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e4a622640 1 -- 172.21.15.122:0/1498854579 >> v1:172.21.15.122:6789/0 conn(0x7f4e4413d8d0 legacy=0x7f4e4406b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:06.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e4a622640 1 -- 172.21.15.122:0/1498854579 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e44146ee0 con 0x7f4e44071580 2024-03-16T20:46:06.805 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e49620640 1 --2- 172.21.15.122:0/1498854579 >> v2:172.21.15.195:3300/0 conn(0x7f4e44146a50 0x7f4e44143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:46:06.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e4a622640 1 --2- 172.21.15.122:0/1498854579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e44071580 0x7f4e441439a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4e4406e420 tx=0x7f4e3c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391b2ceb69d83c73 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:06.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e48e1f640 1 -- 172.21.15.122:0/1498854579 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c0a5880 con 0x7f4e44071580 2024-03-16T20:46:06.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e48e1f640 1 -- 172.21.15.122:0/1498854579 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c0a59e0 con 0x7f4e44071580 2024-03-16T20:46:06.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.798+0000 7f4e48e1f640 1 -- 172.21.15.122:0/1498854579 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c0a5d70 con 0x7f4e44071580 2024-03-16T20:46:06.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 -- 172.21.15.122:0/1498854579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e44071580 msgr2=0x7f4e441439a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:06.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 --2- 172.21.15.122:0/1498854579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e44071580 0x7f4e441439a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4e4406e420 tx=0x7f4e3c01c930 comp rx=0 tx=0).stop 2024-03-16T20:46:06.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 -- 172.21.15.122:0/1498854579 shutdown_connections 2024-03-16T20:46:06.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 --2- 172.21.15.122:0/1498854579 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e44071580 0x7f4e441439a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:06.807 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 --2- 172.21.15.122:0/1498854579 >> v2:172.21.15.195:3300/0 conn(0x7f4e44146a50 0x7f4e44143460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:06.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 -- 172.21.15.122:0/1498854579 >> 172.21.15.122:0/1498854579 conn(0x7f4e440b8860 msgr2=0x7f4e440b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:06.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 -- 172.21.15.122:0/1498854579 shutdown_connections 2024-03-16T20:46:06.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 -- 172.21.15.122:0/1498854579 wait complete. 2024-03-16T20:46:06.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 Processor -- start 2024-03-16T20:46:06.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 -- start start 2024-03-16T20:46:06.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e44071580 0x7f4e441413b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:06.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e4413d8d0 0x7f4e44141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:06.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e441e3020 0x7f4e441e34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:06.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4414d990 con 0x7f4e4413d8d0 2024-03-16T20:46:06.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e441423f0 con 0x7f4e441e3020 2024-03-16T20:46:06.810 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e441426a0 con 0x7f4e44071580 2024-03-16T20:46:06.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e49620640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e4413d8d0 0x7f4e44141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:06.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e49620640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e4413d8d0 0x7f4e44141eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40846/0 (socket says 172.21.15.122:40846) 2024-03-16T20:46:06.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e49620640 1 -- 172.21.15.122:0/1095906817 learned_addr learned my addr 172.21.15.122:0/1095906817 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:06.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4a622640 1 --2- 172.21.15.122:0/1095906817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e441e3020 0x7f4e441e34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:06.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e49e21640 1 --2- 172.21.15.122:0/1095906817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e44071580 0x7f4e441413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:06.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e49620640 1 -- 172.21.15.122:0/1095906817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e44071580 msgr2=0x7f4e441413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:06.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e49620640 1 --2- 172.21.15.122:0/1095906817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e44071580 0x7f4e441413b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:06.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e49620640 1 -- 172.21.15.122:0/1095906817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e441e3020 msgr2=0x7f4e441e34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:06.811 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e49620640 1 --2- 172.21.15.122:0/1095906817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e441e3020 0x7f4e441e34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:06.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e49620640 1 -- 172.21.15.122:0/1095906817 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c009a70 con 0x7f4e4413d8d0 2024-03-16T20:46:06.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4a622640 1 --2- 172.21.15.122:0/1095906817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e441e3020 0x7f4e441e34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:46:06.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e49e21640 1 --2- 172.21.15.122:0/1095906817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e44071580 0x7f4e441413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:46:06.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e49620640 1 --2- 172.21.15.122:0/1095906817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e4413d8d0 0x7f4e44141eb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400d800 tx=0x7f4e3400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:06.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e3a7fc640 1 -- 172.21.15.122:0/1095906817 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e340096c0 con 0x7f4e4413d8d0 2024-03-16T20:46:06.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e3a7fc640 1 -- 172.21.15.122:0/1095906817 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e34009820 con 0x7f4e4413d8d0 2024-03-16T20:46:06.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 -- 172.21.15.122:0/1095906817 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e44142980 con 0x7f4e4413d8d0 2024-03-16T20:46:06.812 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e3a7fc640 1 -- 172.21.15.122:0/1095906817 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e34009bd0 con 0x7f4e4413d8d0 2024-03-16T20:46:06.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.802+0000 7f4e4c085640 1 -- 172.21.15.122:0/1095906817 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e440b7e40 con 0x7f4e4413d8d0 2024-03-16T20:46:06.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.806+0000 7f4e4c085640 1 -- 172.21.15.122:0/1095906817 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e441418f0 con 0x7f4e4413d8d0 2024-03-16T20:46:06.813 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.806+0000 7f4e3a7fc640 1 -- 172.21.15.122:0/1095906817 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f4e34009d30 con 0x7f4e4413d8d0 2024-03-16T20:46:06.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.806+0000 7f4e3a7fc640 1 --2- 172.21.15.122:0/1095906817 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4e28066e70 0x7f4e28069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:06.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.806+0000 7f4e3a7fc640 1 -- 172.21.15.122:0/1095906817 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4e34084db0 con 0x7f4e4413d8d0 2024-03-16T20:46:06.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.806+0000 7f4e49e21640 1 --2- 172.21.15.122:0/1095906817 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4e28066e70 0x7f4e28069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:06.814 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.806+0000 7f4e49e21640 1 --2- 172.21.15.122:0/1095906817 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4e28066e70 0x7f4e28069330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4e400011f0 tx=0x7f4e40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:06.816 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.810+0000 7f4e3a7fc640 1 -- 172.21.15.122:0/1095906817 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f4e34051f30 con 0x7f4e4413d8d0 2024-03-16T20:46:06.976 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.970+0000 7f4e4c085640 1 -- 172.21.15.122:0/1095906817 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4e44071d10 con 0x7f4e28066e70 2024-03-16T20:46:06.978 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.970+0000 7f4e3a7fc640 1 -- 172.21.15.122:0/1095906817 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13374 (secure 0 0 0) 0x7f4e44071d10 con 0x7f4e28066e70 2024-03-16T20:46:06.978 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:46:06.979 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-03-16T20:46:06.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.974+0000 7f4e4c085640 1 -- 172.21.15.122:0/1095906817 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4e28066e70 msgr2=0x7f4e28069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:06.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.974+0000 7f4e4c085640 1 --2- 172.21.15.122:0/1095906817 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4e28066e70 0x7f4e28069330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4e400011f0 tx=0x7f4e40009040 comp rx=0 tx=0).stop 2024-03-16T20:46:06.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.974+0000 7f4e4c085640 1 -- 172.21.15.122:0/1095906817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e4413d8d0 msgr2=0x7f4e44141eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:06.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.974+0000 7f4e4c085640 1 --2- 172.21.15.122:0/1095906817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e4413d8d0 0x7f4e44141eb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400d800 tx=0x7f4e3400dcc0 comp rx=0 tx=0).stop 2024-03-16T20:46:06.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.974+0000 7f4e4c085640 1 -- 172.21.15.122:0/1095906817 shutdown_connections 2024-03-16T20:46:06.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.974+0000 7f4e4c085640 1 --2- 172.21.15.122:0/1095906817 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f4e28066e70 0x7f4e28069330 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:06.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.974+0000 7f4e4c085640 1 --2- 172.21.15.122:0/1095906817 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f4e441e3020 0x7f4e441e34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:06.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.974+0000 7f4e4c085640 1 --2- 172.21.15.122:0/1095906817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4e4413d8d0 0x7f4e44141eb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:06.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.974+0000 7f4e4c085640 1 --2- 172.21.15.122:0/1095906817 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f4e44071580 0x7f4e441413b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:06.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.974+0000 7f4e4c085640 1 -- 172.21.15.122:0/1095906817 >> 172.21.15.122:0/1095906817 conn(0x7f4e440b8860 msgr2=0x7f4e440b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:06.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.974+0000 7f4e4c085640 1 -- 172.21.15.122:0/1095906817 shutdown_connections 2024-03-16T20:46:06.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:06.974+0000 7f4e4c085640 1 -- 172.21.15.122:0/1095906817 wait complete. 2024-03-16T20:46:06.997 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-03-16T20:46:06.598114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236688,"kb_used_data":3864,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738032,"statfs":{"total":773094113280,"available":772851744768,"internally_reserved":0,"allocated":3956736,"data_stored":2231982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10734,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-51,"num_write_kb":-25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001786"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269499+0000","last_change":"2024-03-16T20:44:54.293418+0000","last_active":"2024-03-16T20:45:46.269499+0000","last_peered":"2024-03-16T20:45:46.269499+0000","last_clean":"2024-03-16T20:45:46.269499+0000","last_became_active":"2024-03-16T20:44:54.291347+0000","last_became_peered":"2024-03-16T20:44:54.291347+0000","last_unstale":"2024-03-16T20:45:46.269499+0000","last_undegraded":"2024-03-16T20:45:46.269499+0000","last_fullsized":"2024-03-16T20:45:46.269499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_clean_scrub_stamp":"2024-03-16T20:44:53.077639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:29:18.318494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":933888,"data_stored":719141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":129299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":129355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":128787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":360448,"data_stored":138798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":128753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":360448,"data_stored":138798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":929792,"data_stored":719051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T20:46:06.998 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T20:46:07.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.110+0000 7fcb93d7d640 1 Processor -- start 2024-03-16T20:46:07.118 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.110+0000 7fcb93d7d640 1 -- start start 2024-03-16T20:46:07.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb93d7d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb8c143dd0 0x7fcb8c147be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb93d7d640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8c0b2a70 con 0x7fcb8c0b2690 2024-03-16T20:46:07.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb93d7d640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8c0b4950 con 0x7fcb8c071a20 2024-03-16T20:46:07.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb93d7d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c13e5e0 con 0x7fcb8c143dd0 2024-03-16T20:46:07.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb91b19640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fcb8c0b2690 0x7fcb8c0b41a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:40592/0 (socket says 172.21.15.122:40592) 2024-03-16T20:46:07.119 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb91b19640 1 -- 172.21.15.122:0/2295976691 learned_addr learned my addr 172.21.15.122:0/2295976691 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:46:07.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb91318640 1 --2- 172.21.15.122:0/2295976691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb8c143dd0 0x7fcb8c147be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb90b17640 1 -- 172.21.15.122:0/2295976691 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 69597960 0 0) 0x7fcb8c0b2a70 con 0x7fcb8c0b2690 2024-03-16T20:46:07.121 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb90b17640 1 -- 172.21.15.122:0/2295976691 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb78003530 con 0x7fcb8c0b2690 2024-03-16T20:46:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb90b17640 1 -- 172.21.15.122:0/2295976691 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4106628228 0 0) 0x7fcb78003530 con 0x7fcb8c0b2690 2024-03-16T20:46:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb90b17640 1 -- 172.21.15.122:0/2295976691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb8c143dd0 msgr2=0x7fcb8c147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb90b17640 1 --2- 172.21.15.122:0/2295976691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb8c143dd0 0x7fcb8c147be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb90b17640 1 -- 172.21.15.122:0/2295976691 >> v1:172.21.15.195:6789/0 conn(0x7fcb8c071a20 legacy=0x7fcb8c1445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb90b17640 1 -- 172.21.15.122:0/2295976691 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c1473e0 con 0x7fcb8c0b2690 2024-03-16T20:46:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb90b17640 1 -- 172.21.15.122:0/2295976691 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7fcb88001680 con 0x7fcb8c0b2690 2024-03-16T20:46:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb90b17640 1 -- 172.21.15.122:0/2295976691 >> v1:172.21.15.122:6789/0 conn(0x7fcb8c0b2690 legacy=0x7fcb8c0b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.122 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb90b17640 1 --2- 172.21.15.122:0/2295976691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb8c143dd0 0x7fcb780039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb90b17640 1 --2- 172.21.15.122:0/2295976691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb78003f30 0x7fcb780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb90b17640 1 --2- 172.21.15.122:0/2295976691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb78004d10 0x7fcb78005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb90b17640 1 -- 172.21.15.122:0/2295976691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c13e5e0 con 0x7fcb78004d10 2024-03-16T20:46:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb90b17640 1 -- 172.21.15.122:0/2295976691 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7800ba70 con 0x7fcb78003f30 2024-03-16T20:46:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb90b17640 1 -- 172.21.15.122:0/2295976691 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7800bc30 con 0x7fcb8c143dd0 2024-03-16T20:46:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb9231a640 1 --2- 172.21.15.122:0/2295976691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb78004d10 0x7fcb78005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb91318640 1 --2- 172.21.15.122:0/2295976691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb78003f30 0x7fcb780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb91b19640 1 --2- 172.21.15.122:0/2295976691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb8c143dd0 0x7fcb780039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.123 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb9231a640 1 -- 172.21.15.122:0/2295976691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb8c143dd0 msgr2=0x7fcb780039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb9231a640 1 --2- 172.21.15.122:0/2295976691 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb8c143dd0 0x7fcb780039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb9231a640 1 -- 172.21.15.122:0/2295976691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb78003f30 msgr2=0x7fcb780047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb9231a640 1 --2- 172.21.15.122:0/2295976691 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb78003f30 0x7fcb780047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.124 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.114+0000 7fcb9231a640 1 -- 172.21.15.122:0/2295976691 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb78003530 con 0x7fcb78004d10 2024-03-16T20:46:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb9231a640 1 --2- 172.21.15.122:0/2295976691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb78004d10 0x7fcb78005190 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcb84001e70 tx=0x7fcb8401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a893805a3560314 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb90b17640 1 -- 172.21.15.122:0/2295976691 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84003b80 con 0x7fcb78004d10 2024-03-16T20:46:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb90b17640 1 -- 172.21.15.122:0/2295976691 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb84003ce0 con 0x7fcb78004d10 2024-03-16T20:46:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb90b17640 1 -- 172.21.15.122:0/2295976691 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb840052c0 con 0x7fcb78004d10 2024-03-16T20:46:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb93d7d640 1 -- 172.21.15.122:0/2295976691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb78004d10 msgr2=0x7fcb78005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb93d7d640 1 --2- 172.21.15.122:0/2295976691 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb78004d10 0x7fcb78005190 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fcb84001e70 tx=0x7fcb8401c780 comp rx=0 tx=0).stop 2024-03-16T20:46:07.127 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb9231a640 1 -- 172.21.15.122:0/2295976691 reap_dead start 2024-03-16T20:46:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb93d7d640 1 -- 172.21.15.122:0/2295976691 shutdown_connections 2024-03-16T20:46:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb93d7d640 1 -- 172.21.15.122:0/2295976691 >> 172.21.15.122:0/2295976691 conn(0x7fcb8c0b8860 msgr2=0x7fcb8c0b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb93d7d640 1 -- 172.21.15.122:0/2295976691 shutdown_connections 2024-03-16T20:46:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb93d7d640 1 -- 172.21.15.122:0/2295976691 wait complete. 2024-03-16T20:46:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb93d7d640 1 Processor -- start 2024-03-16T20:46:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb93d7d640 1 -- start start 2024-03-16T20:46:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb93d7d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb78004d10 0x7fcb8c13c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb93d7d640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb8c0ad8b0 0x7fcb8c0add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb93d7d640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb8c0aec50 0x7fcb8c0af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb93d7d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c0ad3b0 con 0x7fcb78004d10 2024-03-16T20:46:07.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb93d7d640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c0af610 con 0x7fcb8c0ad8b0 2024-03-16T20:46:07.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb93d7d640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c0b67e0 con 0x7fcb8c0aec50 2024-03-16T20:46:07.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb91b19640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb78004d10 0x7fcb8c13c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb91b19640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb78004d10 0x7fcb8c13c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40874/0 (socket says 172.21.15.122:40874) 2024-03-16T20:46:07.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb91b19640 1 -- 172.21.15.122:0/93807900 learned_addr learned my addr 172.21.15.122:0/93807900 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:07.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb91318640 1 --2- 172.21.15.122:0/93807900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb8c0ad8b0 0x7fcb8c0add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.118+0000 7fcb9231a640 1 --2- 172.21.15.122:0/93807900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb8c0aec50 0x7fcb8c0af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.129 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.122+0000 7fcb91b19640 1 -- 172.21.15.122:0/93807900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb8c0aec50 msgr2=0x7fcb8c0af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.122+0000 7fcb91b19640 1 --2- 172.21.15.122:0/93807900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb8c0aec50 0x7fcb8c0af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.122+0000 7fcb91b19640 1 -- 172.21.15.122:0/93807900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb8c0ad8b0 msgr2=0x7fcb8c0add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.122+0000 7fcb91b19640 1 --2- 172.21.15.122:0/93807900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb8c0ad8b0 0x7fcb8c0add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.122+0000 7fcb91b19640 1 -- 172.21.15.122:0/93807900 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c0ae350 con 0x7fcb78004d10 2024-03-16T20:46:07.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.122+0000 7fcb91b19640 1 --2- 172.21.15.122:0/93807900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb78004d10 0x7fcb8c13c100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcb88007320 tx=0x7fcb88007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:07.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.122+0000 7fcb827fc640 1 -- 172.21.15.122:0/93807900 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8800f940 con 0x7fcb78004d10 2024-03-16T20:46:07.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.122+0000 7fcb827fc640 1 -- 172.21.15.122:0/93807900 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8800fac0 con 0x7fcb78004d10 2024-03-16T20:46:07.130 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.122+0000 7fcb93d7d640 1 -- 172.21.15.122:0/93807900 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8c0acce0 con 0x7fcb78004d10 2024-03-16T20:46:07.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.122+0000 7fcb827fc640 1 -- 172.21.15.122:0/93807900 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8800fe70 con 0x7fcb78004d10 2024-03-16T20:46:07.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.122+0000 7fcb93d7d640 1 -- 172.21.15.122:0/93807900 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8c0b3090 con 0x7fcb78004d10 2024-03-16T20:46:07.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.122+0000 7fcb93d7d640 1 -- 172.21.15.122:0/93807900 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8c075280 con 0x7fcb78004d10 2024-03-16T20:46:07.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.122+0000 7fcb827fc640 1 -- 172.21.15.122:0/93807900 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fcb8801d460 con 0x7fcb78004d10 2024-03-16T20:46:07.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.122+0000 7fcb827fc640 1 --2- 172.21.15.122:0/93807900 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcb70066e40 0x7fcb70069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.122+0000 7fcb827fc640 1 -- 172.21.15.122:0/93807900 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcb88084860 con 0x7fcb78004d10 2024-03-16T20:46:07.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.122+0000 7fcb91318640 1 --2- 172.21.15.122:0/93807900 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcb70066e40 0x7fcb70069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.122+0000 7fcb91318640 1 --2- 172.21.15.122:0/93807900 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcb70066e40 0x7fcb70069300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c0721c0 tx=0x7fcb7c009190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:07.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.126+0000 7fcb827fc640 1 -- 172.21.15.122:0/93807900 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fcb880519e0 con 0x7fcb78004d10 2024-03-16T20:46:07.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.282+0000 7fcb93d7d640 1 -- 172.21.15.122:0/93807900 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb8c0ad210 con 0x7fcb70066e40 2024-03-16T20:46:07.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.282+0000 7fcb827fc640 1 -- 172.21.15.122:0/93807900 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13374 (secure 0 0 0) 0x7fcb7c019070 con 0x7fcb70066e40 2024-03-16T20:46:07.290 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:46:07.291 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-03-16T20:46:07.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.286+0000 7fcb93d7d640 1 -- 172.21.15.122:0/93807900 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcb70066e40 msgr2=0x7fcb70069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.286+0000 7fcb93d7d640 1 --2- 172.21.15.122:0/93807900 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcb70066e40 0x7fcb70069300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c0721c0 tx=0x7fcb7c009190 comp rx=0 tx=0).stop 2024-03-16T20:46:07.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.286+0000 7fcb93d7d640 1 -- 172.21.15.122:0/93807900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb78004d10 msgr2=0x7fcb8c13c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.286+0000 7fcb93d7d640 1 --2- 172.21.15.122:0/93807900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb78004d10 0x7fcb8c13c100 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fcb88007320 tx=0x7fcb88007720 comp rx=0 tx=0).stop 2024-03-16T20:46:07.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.286+0000 7fcb93d7d640 1 -- 172.21.15.122:0/93807900 shutdown_connections 2024-03-16T20:46:07.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.286+0000 7fcb93d7d640 1 --2- 172.21.15.122:0/93807900 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fcb70066e40 0x7fcb70069300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.293 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.286+0000 7fcb93d7d640 1 --2- 172.21.15.122:0/93807900 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fcb8c0aec50 0x7fcb8c0af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.286+0000 7fcb93d7d640 1 --2- 172.21.15.122:0/93807900 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fcb8c0ad8b0 0x7fcb8c0add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.286+0000 7fcb93d7d640 1 --2- 172.21.15.122:0/93807900 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb78004d10 0x7fcb8c13c100 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.286+0000 7fcb93d7d640 1 -- 172.21.15.122:0/93807900 >> 172.21.15.122:0/93807900 conn(0x7fcb8c0b8860 msgr2=0x7fcb8c0b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:07.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.286+0000 7fcb93d7d640 1 -- 172.21.15.122:0/93807900 shutdown_connections 2024-03-16T20:46:07.294 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.286+0000 7fcb93d7d640 1 -- 172.21.15.122:0/93807900 wait complete. 2024-03-16T20:46:07.309 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-03-16T20:46:06.598114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236688,"kb_used_data":3864,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738032,"statfs":{"total":773094113280,"available":772851744768,"internally_reserved":0,"allocated":3956736,"data_stored":2231982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10734,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-51,"num_write_kb":-25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001786"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269499+0000","last_change":"2024-03-16T20:44:54.293418+0000","last_active":"2024-03-16T20:45:46.269499+0000","last_peered":"2024-03-16T20:45:46.269499+0000","last_clean":"2024-03-16T20:45:46.269499+0000","last_became_active":"2024-03-16T20:44:54.291347+0000","last_became_peered":"2024-03-16T20:44:54.291347+0000","last_unstale":"2024-03-16T20:45:46.269499+0000","last_undegraded":"2024-03-16T20:45:46.269499+0000","last_fullsized":"2024-03-16T20:45:46.269499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_clean_scrub_stamp":"2024-03-16T20:44:53.077639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:29:18.318494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":933888,"data_stored":719141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":129299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":129355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":128787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":360448,"data_stored":138798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":128753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":360448,"data_stored":138798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":929792,"data_stored":719051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T20:46:07.309 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-03-16T20:46:07.310 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T20:46:07.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.418+0000 7fc5e65f9640 1 Processor -- start 2024-03-16T20:46:07.425 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.418+0000 7fc5e65f9640 1 -- start start 2024-03-16T20:46:07.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5e65f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e00baca0 0x7fc5e0144e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5e65f9640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc5e0148ec0 0x7fc5e00b1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5e65f9640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e0149330 con 0x7fc5e013eb30 2024-03-16T20:46:07.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5e65f9640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5e00b06b0 con 0x7fc5e0148ec0 2024-03-16T20:46:07.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5e65f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e00b1cd0 con 0x7fc5e00baca0 2024-03-16T20:46:07.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5df7fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc5e0148ec0 0x7fc5e00b1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5e4b96640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fc5e013eb30 0x7fc5e013ef30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:40608/0 (socket says 172.21.15.122:40608) 2024-03-16T20:46:07.427 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5e4b96640 1 -- 172.21.15.122:0/1228109147 learned_addr learned my addr 172.21.15.122:0/1228109147 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:46:07.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5dffff640 1 --2- 172.21.15.122:0/1228109147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e00baca0 0x7fc5e0144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5df7fe640 1 -- 172.21.15.122:0/1228109147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e00baca0 msgr2=0x7fc5e0144e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:07.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5df7fe640 1 --2- 172.21.15.122:0/1228109147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e00baca0 0x7fc5e0144e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5df7fe640 1 -- 172.21.15.122:0/1228109147 >> v1:172.21.15.122:6789/0 conn(0x7fc5e013eb30 legacy=0x7fc5e013ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5df7fe640 1 -- 172.21.15.122:0/1228109147 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e00bb080 con 0x7fc5e0148ec0 2024-03-16T20:46:07.428 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5dffff640 1 --2- 172.21.15.122:0/1228109147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e00baca0 0x7fc5e0144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:07.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5df7fe640 1 --2- 172.21.15.122:0/1228109147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5e0148ec0 0x7fc5e00b1790 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40097d0 tx=0x7fc5d401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=358cc19901c1fb74 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:07.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5deffd640 1 -- 172.21.15.122:0/1228109147 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d40ab600 con 0x7fc5e0148ec0 2024-03-16T20:46:07.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5deffd640 1 -- 172.21.15.122:0/1228109147 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d40a7050 con 0x7fc5e0148ec0 2024-03-16T20:46:07.429 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5deffd640 1 -- 172.21.15.122:0/1228109147 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d40ab980 con 0x7fc5e0148ec0 2024-03-16T20:46:07.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5e65f9640 1 -- 172.21.15.122:0/1228109147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5e0148ec0 msgr2=0x7fc5e00b1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5e65f9640 1 --2- 172.21.15.122:0/1228109147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5e0148ec0 0x7fc5e00b1790 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40097d0 tx=0x7fc5d401c3f0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.430 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5e65f9640 1 -- 172.21.15.122:0/1228109147 shutdown_connections 2024-03-16T20:46:07.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5e65f9640 1 --2- 172.21.15.122:0/1228109147 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5e0148ec0 0x7fc5e00b1790 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5e65f9640 1 --2- 172.21.15.122:0/1228109147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e00baca0 0x7fc5e0144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5e65f9640 1 -- 172.21.15.122:0/1228109147 >> 172.21.15.122:0/1228109147 conn(0x7fc5e006ee40 msgr2=0x7fc5e006f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:07.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5e65f9640 1 -- 172.21.15.122:0/1228109147 shutdown_connections 2024-03-16T20:46:07.431 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.422+0000 7fc5e65f9640 1 -- 172.21.15.122:0/1228109147 wait complete. 2024-03-16T20:46:07.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e65f9640 1 Processor -- start 2024-03-16T20:46:07.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e65f9640 1 -- start start 2024-03-16T20:46:07.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e65f9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e00baca0 0x7fc5e00b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e65f9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5e013eb30 0x7fc5e013fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e65f9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5e0148ec0 0x7fc5e0141960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e65f9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e01405a0 con 0x7fc5e0148ec0 2024-03-16T20:46:07.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e65f9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e0140370 con 0x7fc5e013eb30 2024-03-16T20:46:07.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e65f9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e0140f70 con 0x7fc5e00baca0 2024-03-16T20:46:07.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e4b96640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5e0148ec0 0x7fc5e0141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e4b96640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5e0148ec0 0x7fc5e0141960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40898/0 (socket says 172.21.15.122:40898) 2024-03-16T20:46:07.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e4b96640 1 -- 172.21.15.122:0/281595063 learned_addr learned my addr 172.21.15.122:0/281595063 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:07.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5dffff640 1 --2- 172.21.15.122:0/281595063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e00baca0 0x7fc5e00b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5df7fe640 1 --2- 172.21.15.122:0/281595063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5e013eb30 0x7fc5e013fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e4b96640 1 -- 172.21.15.122:0/281595063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e00baca0 msgr2=0x7fc5e00b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e4b96640 1 --2- 172.21.15.122:0/281595063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e00baca0 0x7fc5e00b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e4b96640 1 -- 172.21.15.122:0/281595063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5e013eb30 msgr2=0x7fc5e013fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e4b96640 1 --2- 172.21.15.122:0/281595063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5e013eb30 0x7fc5e013fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e4b96640 1 -- 172.21.15.122:0/281595063 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d40093b0 con 0x7fc5e0148ec0 2024-03-16T20:46:07.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5dffff640 1 --2- 172.21.15.122:0/281595063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e00baca0 0x7fc5e00b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:07.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e4b96640 1 --2- 172.21.15.122:0/281595063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5e0148ec0 0x7fc5e0141960 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800cbf0 tx=0x7fc5d800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:07.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5dcff9640 1 -- 172.21.15.122:0/281595063 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d800ad40 con 0x7fc5e0148ec0 2024-03-16T20:46:07.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5dcff9640 1 -- 172.21.15.122:0/281595063 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d8012400 con 0x7fc5e0148ec0 2024-03-16T20:46:07.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e65f9640 1 -- 172.21.15.122:0/281595063 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e0142470 con 0x7fc5e0148ec0 2024-03-16T20:46:07.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5dcff9640 1 -- 172.21.15.122:0/281595063 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d800aea0 con 0x7fc5e0148ec0 2024-03-16T20:46:07.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e65f9640 1 -- 172.21.15.122:0/281595063 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e0141400 con 0x7fc5e0148ec0 2024-03-16T20:46:07.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5e65f9640 1 -- 172.21.15.122:0/281595063 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e00677b0 con 0x7fc5e0148ec0 2024-03-16T20:46:07.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.426+0000 7fc5dcff9640 1 -- 172.21.15.122:0/281595063 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fc5d8007070 con 0x7fc5e0148ec0 2024-03-16T20:46:07.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.430+0000 7fc5dcff9640 1 --2- 172.21.15.122:0/281595063 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc5cc066e40 0x7fc5cc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.430+0000 7fc5dcff9640 1 -- 172.21.15.122:0/281595063 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc5d80845b0 con 0x7fc5e0148ec0 2024-03-16T20:46:07.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.430+0000 7fc5dffff640 1 --2- 172.21.15.122:0/281595063 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc5cc066e40 0x7fc5cc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.437 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.430+0000 7fc5dffff640 1 --2- 172.21.15.122:0/281595063 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc5cc066e40 0x7fc5cc069300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0008930 tx=0x7fc5d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:07.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.434+0000 7fc5dcff9640 1 -- 172.21.15.122:0/281595063 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc5d8051740 con 0x7fc5e0148ec0 2024-03-16T20:46:07.596 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.590+0000 7fc5e65f9640 1 -- 172.21.15.122:0/281595063 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc5e00b1750 con 0x7fc5cc066e40 2024-03-16T20:46:07.598 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.590+0000 7fc5dcff9640 1 -- 172.21.15.122:0/281595063 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13374 (secure 0 0 0) 0x7fc5e00b1750 con 0x7fc5cc066e40 2024-03-16T20:46:07.598 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:46:07.599 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-03-16T20:46:07.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.594+0000 7fc5e65f9640 1 -- 172.21.15.122:0/281595063 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc5cc066e40 msgr2=0x7fc5cc069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.594+0000 7fc5e65f9640 1 --2- 172.21.15.122:0/281595063 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc5cc066e40 0x7fc5cc069300 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0008930 tx=0x7fc5d0008040 comp rx=0 tx=0).stop 2024-03-16T20:46:07.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.594+0000 7fc5e65f9640 1 -- 172.21.15.122:0/281595063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5e0148ec0 msgr2=0x7fc5e0141960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.594+0000 7fc5e65f9640 1 --2- 172.21.15.122:0/281595063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5e0148ec0 0x7fc5e0141960 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800cbf0 tx=0x7fc5d800a430 comp rx=0 tx=0).stop 2024-03-16T20:46:07.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.594+0000 7fc5e65f9640 1 -- 172.21.15.122:0/281595063 shutdown_connections 2024-03-16T20:46:07.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.594+0000 7fc5e65f9640 1 --2- 172.21.15.122:0/281595063 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc5cc066e40 0x7fc5cc069300 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.594+0000 7fc5e65f9640 1 --2- 172.21.15.122:0/281595063 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5e0148ec0 0x7fc5e0141960 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.601 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.594+0000 7fc5e65f9640 1 --2- 172.21.15.122:0/281595063 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc5e013eb30 0x7fc5e013fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.594+0000 7fc5e65f9640 1 --2- 172.21.15.122:0/281595063 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc5e00baca0 0x7fc5e00b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.594+0000 7fc5e65f9640 1 -- 172.21.15.122:0/281595063 >> 172.21.15.122:0/281595063 conn(0x7fc5e006ee40 msgr2=0x7fc5e0142d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.594+0000 7fc5e65f9640 1 -- 172.21.15.122:0/281595063 shutdown_connections 2024-03-16T20:46:07.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.594+0000 7fc5e65f9640 1 -- 172.21.15.122:0/281595063 wait complete. 2024-03-16T20:46:07.616 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-03-16T20:46:06.598114+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236688,"kb_used_data":3864,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738032,"statfs":{"total":773094113280,"available":772851744768,"internally_reserved":0,"allocated":3956736,"data_stored":2231982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10734,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-51,"num_write_kb":-25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001786"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":187,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-03-16T20:45:46.269499+0000","last_change":"2024-03-16T20:44:54.293418+0000","last_active":"2024-03-16T20:45:46.269499+0000","last_peered":"2024-03-16T20:45:46.269499+0000","last_clean":"2024-03-16T20:45:46.269499+0000","last_became_active":"2024-03-16T20:44:54.291347+0000","last_became_peered":"2024-03-16T20:44:54.291347+0000","last_unstale":"2024-03-16T20:45:46.269499+0000","last_undegraded":"2024-03-16T20:45:46.269499+0000","last_fullsized":"2024-03-16T20:45:46.269499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_clean_scrub_stamp":"2024-03-16T20:44:53.077639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:29:18.318494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32096,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":933888,"data_stored":719141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":129299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":129355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31524,"kb_used_data":336,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340316,"statfs":{"total":96636764160,"available":96604483584,"internally_reserved":0,"allocated":344064,"data_stored":128787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":360448,"data_stored":138798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":128753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":360448,"data_stored":138798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":929792,"data_stored":719051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T20:46:07.617 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-16T20:46:07.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.730+0000 7f44bd9b5640 1 Processor -- start 2024-03-16T20:46:07.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.734+0000 7f44bd9b5640 1 -- start start 2024-03-16T20:46:07.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.734+0000 7f44bd9b5640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f44b80ba850 0x7f44b81471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.734+0000 7f44bd9b5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44b809ba00 0x7f44b809b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.734+0000 7f44bd9b5640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b809aee0 con 0x7f44b8147e90 2024-03-16T20:46:07.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.734+0000 7f44bd9b5640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b80bb1d0 con 0x7f44b80ba850 2024-03-16T20:46:07.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.734+0000 7f44bd9b5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b80b9280 con 0x7f44b809ba00 2024-03-16T20:46:07.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.734+0000 7f44b67fc640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f44b80ba850 0x7f44b81471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.734+0000 7f44b67fc640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f44b80ba850 0x7f44b81471e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40914/0 (socket says 172.21.15.122:40914) 2024-03-16T20:46:07.740 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.734+0000 7f44b67fc640 1 -- 172.21.15.122:0/1907010122 learned_addr learned my addr 172.21.15.122:0/1907010122 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:07.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.734+0000 7f44b77fe640 1 --2- 172.21.15.122:0/1907010122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44b809ba00 0x7f44b809b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.734+0000 7f44b67fc640 1 -- 172.21.15.122:0/1907010122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44b809ba00 msgr2=0x7f44b809b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:07.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.734+0000 7f44b67fc640 1 --2- 172.21.15.122:0/1907010122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44b809ba00 0x7f44b809b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.734+0000 7f44b67fc640 1 -- 172.21.15.122:0/1907010122 >> v1:172.21.15.195:6789/0 conn(0x7f44b8147e90 legacy=0x7f44b80b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.734+0000 7f44b67fc640 1 -- 172.21.15.122:0/1907010122 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b809ab70 con 0x7f44b80ba850 2024-03-16T20:46:07.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44b67fc640 1 --2- 172.21.15.122:0/1907010122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44b80ba850 0x7f44b81471e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009870 tx=0x7f44ac01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b0545528207d38 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:07.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44b5ffb640 1 -- 172.21.15.122:0/1907010122 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac004230 con 0x7f44b80ba850 2024-03-16T20:46:07.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44b5ffb640 1 -- 172.21.15.122:0/1907010122 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ac0a7050 con 0x7f44b80ba850 2024-03-16T20:46:07.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44b5ffb640 1 -- 172.21.15.122:0/1907010122 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ac003620 con 0x7f44b80ba850 2024-03-16T20:46:07.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44bd9b5640 1 -- 172.21.15.122:0/1907010122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44b80ba850 msgr2=0x7f44b81471e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44bd9b5640 1 --2- 172.21.15.122:0/1907010122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44b80ba850 0x7f44b81471e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009870 tx=0x7f44ac01c440 comp rx=0 tx=0).stop 2024-03-16T20:46:07.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44bd9b5640 1 -- 172.21.15.122:0/1907010122 shutdown_connections 2024-03-16T20:46:07.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44bd9b5640 1 --2- 172.21.15.122:0/1907010122 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44b809ba00 0x7f44b809b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44bd9b5640 1 --2- 172.21.15.122:0/1907010122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44b80ba850 0x7f44b81471e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44bd9b5640 1 -- 172.21.15.122:0/1907010122 >> 172.21.15.122:0/1907010122 conn(0x7f44b80bca50 msgr2=0x7f44b80bce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:07.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44bd9b5640 1 -- 172.21.15.122:0/1907010122 shutdown_connections 2024-03-16T20:46:07.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44bd9b5640 1 -- 172.21.15.122:0/1907010122 wait complete. 2024-03-16T20:46:07.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44bd9b5640 1 Processor -- start 2024-03-16T20:46:07.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44bd9b5640 1 -- start start 2024-03-16T20:46:07.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44bd9b5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44b809ba00 0x7f44b809d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44bd9b5640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44b8147e90 0x7f44b81e2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44bd9b5640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44b81e3000 0x7f44b80b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44bd9b5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b80b6c10 con 0x7f44b809ba00 2024-03-16T20:46:07.747 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44bd9b5640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b80b5440 con 0x7f44b8147e90 2024-03-16T20:46:07.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44bd9b5640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b80b6780 con 0x7f44b81e3000 2024-03-16T20:46:07.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44b6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44b809ba00 0x7f44b809d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44b6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44b809ba00 0x7f44b809d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:40926/0 (socket says 172.21.15.122:40926) 2024-03-16T20:46:07.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44b6ffd640 1 -- 172.21.15.122:0/2425298655 learned_addr learned my addr 172.21.15.122:0/2425298655 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:07.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44b67fc640 1 --2- 172.21.15.122:0/2425298655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44b8147e90 0x7f44b81e2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44b77fe640 1 --2- 172.21.15.122:0/2425298655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44b81e3000 0x7f44b80b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44b6ffd640 1 -- 172.21.15.122:0/2425298655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44b81e3000 msgr2=0x7f44b80b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44b6ffd640 1 --2- 172.21.15.122:0/2425298655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44b81e3000 0x7f44b80b3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.748 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44b6ffd640 1 -- 172.21.15.122:0/2425298655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44b8147e90 msgr2=0x7f44b81e2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44b6ffd640 1 --2- 172.21.15.122:0/2425298655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44b8147e90 0x7f44b81e2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44b6ffd640 1 -- 172.21.15.122:0/2425298655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ac0094d0 con 0x7f44b809ba00 2024-03-16T20:46:07.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.738+0000 7f44b77fe640 1 --2- 172.21.15.122:0/2425298655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44b81e3000 0x7f44b80b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:07.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.742+0000 7f44b6ffd640 1 --2- 172.21.15.122:0/2425298655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44b809ba00 0x7f44b809d7e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f44b000c7c0 tx=0x7f44b000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:07.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.742+0000 7f449ffff640 1 -- 172.21.15.122:0/2425298655 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0006d60 con 0x7f44b809ba00 2024-03-16T20:46:07.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.742+0000 7f449ffff640 1 -- 172.21.15.122:0/2425298655 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b0003980 con 0x7f44b809ba00 2024-03-16T20:46:07.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.742+0000 7f449ffff640 1 -- 172.21.15.122:0/2425298655 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0003d30 con 0x7f44b809ba00 2024-03-16T20:46:07.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.742+0000 7f44bd9b5640 1 -- 172.21.15.122:0/2425298655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b80a09e0 con 0x7f44b809ba00 2024-03-16T20:46:07.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.742+0000 7f44bd9b5640 1 -- 172.21.15.122:0/2425298655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b80a0700 con 0x7f44b809ba00 2024-03-16T20:46:07.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.742+0000 7f44bd9b5640 1 -- 172.21.15.122:0/2425298655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b80b6520 con 0x7f44b809ba00 2024-03-16T20:46:07.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.742+0000 7f449ffff640 1 -- 172.21.15.122:0/2425298655 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f44b000a850 con 0x7f44b809ba00 2024-03-16T20:46:07.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.742+0000 7f449ffff640 1 --2- 172.21.15.122:0/2425298655 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f44a4066e40 0x7f44a4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:07.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.742+0000 7f449ffff640 1 -- 172.21.15.122:0/2425298655 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f44b0084350 con 0x7f44b809ba00 2024-03-16T20:46:07.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.742+0000 7f44b67fc640 1 --2- 172.21.15.122:0/2425298655 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f44a4066e40 0x7f44a4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:07.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.742+0000 7f44b67fc640 1 --2- 172.21.15.122:0/2425298655 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f44a4066e40 0x7f44a4069300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f44ac0ad000 tx=0x7f44ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:07.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.746+0000 7f449ffff640 1 -- 172.21.15.122:0/2425298655 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f44b00514d0 con 0x7f44b809ba00 2024-03-16T20:46:07.911 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.906+0000 7f44bd9b5640 1 -- 172.21.15.122:0/2425298655 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44b8075280 con 0x7f44b809ba00 2024-03-16T20:46:07.913 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.906+0000 7f449ffff640 1 -- 172.21.15.122:0/2425298655 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+13058 (secure 0 0 0) 0x7f44b0054b90 con 0x7f44b809ba00 2024-03-16T20:46:07.913 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:46:07.913 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":33,"fsid":"ea4c964c-3481-413d-a556-a3fc2950fdd6","created":"2024-03-16T20:44:40.043973+0000","modified":"2024-03-16T20:46:05.030172+0000","last_up_change":"2024-03-16T20:44:51.072272+0000","last_in_change":"2024-03-16T20:44:42.665892+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-16T20:44:52.653523+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"bf3aff4c-c53f-4a6e-a538-7550585bb0a0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6809","nonce":4193825355}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6811","nonce":4193825355}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6815","nonce":4193825355}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":4193825355},{"type":"v1","addr":"172.21.15.122:6813","nonce":4193825355}]},"public_addr":"172.21.15.122:6809/4193825355","cluster_addr":"172.21.15.122:6811/4193825355","heartbeat_back_addr":"172.21.15.122:6815/4193825355","heartbeat_front_addr":"172.21.15.122:6813/4193825355","state":["exists","up"]},{"osd":1,"uuid":"fe6ef968-e4d8-46fd-8aa5-4d47be5b9a3b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6801","nonce":685793600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6803","nonce":685793600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6807","nonce":685793600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":685793600},{"type":"v1","addr":"172.21.15.122:6805","nonce":685793600}]},"public_addr":"172.21.15.122:6801/685793600","cluster_addr":"172.21.15.122:6803/685793600","heartbeat_back_addr":"172.21.15.122:6807/685793600","heartbeat_front_addr":"172.21.15.122:6805/685793600","state":["exists","up"]},{"osd":2,"uuid":"367a357b-c342-44c6-8155-fdab43bfb947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6819","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6822","nonce":2377812328}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6826","nonce":2377812328}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6831","nonce":2377812328}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":2377812328},{"type":"v1","addr":"172.21.15.122:6829","nonce":2377812328}]},"public_addr":"172.21.15.122:6822/2377812328","cluster_addr":"172.21.15.122:6826/2377812328","heartbeat_back_addr":"172.21.15.122:6831/2377812328","heartbeat_front_addr":"172.21.15.122:6829/2377812328","state":["exists","up"]},{"osd":3,"uuid":"3587f37a-c824-4dea-a154-297942ff1e1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6817","nonce":362583029}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6820","nonce":362583029}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6825","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6827","nonce":362583029}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6821","nonce":362583029},{"type":"v1","addr":"172.21.15.122:6823","nonce":362583029}]},"public_addr":"172.21.15.122:6817/362583029","cluster_addr":"172.21.15.122:6820/362583029","heartbeat_back_addr":"172.21.15.122:6827/362583029","heartbeat_front_addr":"172.21.15.122:6823/362583029","state":["exists","up"]},{"osd":4,"uuid":"d7fa79af-de38-4c3a-96e6-e97126092629","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6807","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6809","nonce":3059274194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6812","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6815","nonce":3059274194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6822","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6823","nonce":3059274194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6819","nonce":3059274194},{"type":"v1","addr":"172.21.15.195:6821","nonce":3059274194}]},"public_addr":"172.21.15.195:6809/3059274194","cluster_addr":"172.21.15.195:6815/3059274194","heartbeat_back_addr":"172.21.15.195:6823/3059274194","heartbeat_front_addr":"172.21.15.195:6821/3059274194","state":["exists","up"]},{"osd":5,"uuid":"f5d2b51c-29b9-40be-bcb7-134c4e1343b2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6800","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6801","nonce":1568352129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6802","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6803","nonce":1568352129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6806","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6808","nonce":1568352129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6804","nonce":1568352129},{"type":"v1","addr":"172.21.15.195:6805","nonce":1568352129}]},"public_addr":"172.21.15.195:6801/1568352129","cluster_addr":"172.21.15.195:6803/1568352129","heartbeat_back_addr":"172.21.15.195:6808/1568352129","heartbeat_front_addr":"172.21.15.195:6805/1568352129","state":["exists","up"]},{"osd":6,"uuid":"4d70db10-084a-4b65-b9b9-759620154490","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6810","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6811","nonce":1730924554}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6813","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6814","nonce":1730924554}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6818","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6820","nonce":1730924554}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6816","nonce":1730924554},{"type":"v1","addr":"172.21.15.195:6817","nonce":1730924554}]},"public_addr":"172.21.15.195:6811/1730924554","cluster_addr":"172.21.15.195:6814/1730924554","heartbeat_back_addr":"172.21.15.195:6820/1730924554","heartbeat_front_addr":"172.21.15.195:6817/1730924554","state":["exists","up"]},{"osd":7,"uuid":"ac0af635-e34b-4876-a01d-d9c98833385e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6824","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6825","nonce":3035467983}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6826","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6827","nonce":3035467983}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6830","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6831","nonce":3035467983}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6828","nonce":3035467983},{"type":"v1","addr":"172.21.15.195:6829","nonce":3035467983}]},"public_addr":"172.21.15.195:6825/3035467983","cluster_addr":"172.21.15.195:6827/3035467983","heartbeat_back_addr":"172.21.15.195:6831/3035467983","heartbeat_front_addr":"172.21.15.195:6829/3035467983","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.229916+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:47.902580+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.303532+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.185794+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.109393+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.155881+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.248186+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-16T20:44:48.188819+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.122:6833/1841055069":"2024-03-17T20:46:01.699106+0000","172.21.15.122:6832/1841055069":"2024-03-17T20:46:01.699106+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-16T20:46:07.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.910+0000 7f44bd9b5640 1 -- 172.21.15.122:0/2425298655 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f44a4066e40 msgr2=0x7f44a4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.915 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.910+0000 7f44bd9b5640 1 --2- 172.21.15.122:0/2425298655 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f44a4066e40 0x7f44a4069300 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f44ac0ad000 tx=0x7f44ac009040 comp rx=0 tx=0).stop 2024-03-16T20:46:07.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.910+0000 7f44bd9b5640 1 -- 172.21.15.122:0/2425298655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44b809ba00 msgr2=0x7f44b809d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:07.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.910+0000 7f44bd9b5640 1 --2- 172.21.15.122:0/2425298655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44b809ba00 0x7f44b809d7e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f44b000c7c0 tx=0x7f44b000cc80 comp rx=0 tx=0).stop 2024-03-16T20:46:07.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.910+0000 7f44bd9b5640 1 -- 172.21.15.122:0/2425298655 shutdown_connections 2024-03-16T20:46:07.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.910+0000 7f44bd9b5640 1 --2- 172.21.15.122:0/2425298655 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f44a4066e40 0x7f44a4069300 secure :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f44ac0ad000 tx=0x7f44ac009040 comp rx=0 tx=0).stop 2024-03-16T20:46:07.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.910+0000 7f44bd9b5640 1 --2- 172.21.15.122:0/2425298655 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f44b81e3000 0x7f44b80b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.910+0000 7f44bd9b5640 1 --2- 172.21.15.122:0/2425298655 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f44b8147e90 0x7f44b81e2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.910+0000 7f44bd9b5640 1 --2- 172.21.15.122:0/2425298655 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f44b809ba00 0x7f44b809d7e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:07.916 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.910+0000 7f44bd9b5640 1 -- 172.21.15.122:0/2425298655 >> 172.21.15.122:0/2425298655 conn(0x7f44b80bca50 msgr2=0x7f44b8139a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:07.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.910+0000 7f44bd9b5640 1 -- 172.21.15.122:0/2425298655 shutdown_connections 2024-03-16T20:46:07.917 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:07.910+0000 7f44bd9b5640 1 -- 172.21.15.122:0/2425298655 wait complete. 2024-03-16T20:46:08.932 INFO:tasks.ceph:Scrubbing osd.0 2024-03-16T20:46:08.933 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-03-16T20:46:09.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f3627b18640 1 Processor -- start 2024-03-16T20:46:09.068 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f3627b18640 1 -- start start 2024-03-16T20:46:09.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f3627b18640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f362006b1f0 0x7f36200b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.071 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f3627b18640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f362006f6f0 0x7f3620141270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f3627b18640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3620147ca0 con 0x7f36201417b0 2024-03-16T20:46:09.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f3627b18640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362006b5d0 con 0x7f362006b1f0 2024-03-16T20:46:09.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f3627b18640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3620147650 con 0x7f362006f6f0 2024-03-16T20:46:09.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36260b5640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f36201417b0 0x7f362014a870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:42128/0 (socket says 172.21.15.122:42128) 2024-03-16T20:46:09.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36260b5640 1 -- 172.21.15.122:0/3733242248 learned_addr learned my addr 172.21.15.122:0/3733242248 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:46:09.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36258b4640 1 --2- 172.21.15.122:0/3733242248 >> v2:172.21.15.195:3300/0 conn(0x7f362006b1f0 0x7f36200b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36250b3640 1 --2- 172.21.15.122:0/3733242248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f362006f6f0 0x7f3620141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36248b2640 1 -- 172.21.15.122:0/3733242248 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612133211 0 0) 0x7f3620147ca0 con 0x7f36201417b0 2024-03-16T20:46:09.072 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36248b2640 1 -- 172.21.15.122:0/3733242248 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f360c003530 con 0x7f36201417b0 2024-03-16T20:46:09.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36248b2640 1 -- 172.21.15.122:0/3733242248 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2825109793 0 0) 0x7f360c003530 con 0x7f36201417b0 2024-03-16T20:46:09.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36248b2640 1 -- 172.21.15.122:0/3733242248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f362006f6f0 msgr2=0x7f3620141270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:09.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36248b2640 1 --2- 172.21.15.122:0/3733242248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f362006f6f0 0x7f3620141270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36248b2640 1 -- 172.21.15.122:0/3733242248 >> v2:172.21.15.195:3300/0 conn(0x7f362006b1f0 msgr2=0x7f36200b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:09.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36248b2640 1 --2- 172.21.15.122:0/3733242248 >> v2:172.21.15.195:3300/0 conn(0x7f362006b1f0 0x7f36200b7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36248b2640 1 -- 172.21.15.122:0/3733242248 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3620147930 con 0x7f36201417b0 2024-03-16T20:46:09.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36248b2640 1 -- 172.21.15.122:0/3733242248 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f3618001a00 con 0x7f36201417b0 2024-03-16T20:46:09.073 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36258b4640 1 --2- 172.21.15.122:0/3733242248 >> v2:172.21.15.195:3300/0 conn(0x7f362006b1f0 0x7f36200b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:46:09.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36248b2640 1 -- 172.21.15.122:0/3733242248 >> v1:172.21.15.122:6789/0 conn(0x7f36201417b0 legacy=0x7f362014a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36248b2640 1 --2- 172.21.15.122:0/3733242248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f362006f6f0 0x7f360c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36248b2640 1 --2- 172.21.15.122:0/3733242248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f360c003f00 0x7f360c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36248b2640 1 --2- 172.21.15.122:0/3733242248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f360c004d10 0x7f360c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36248b2640 1 -- 172.21.15.122:0/3733242248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362006b5d0 con 0x7f360c003f00 2024-03-16T20:46:09.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36248b2640 1 -- 172.21.15.122:0/3733242248 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3620147650 con 0x7f360c004d10 2024-03-16T20:46:09.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.062+0000 7f36248b2640 1 -- 172.21.15.122:0/3733242248 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360c00b9e0 con 0x7f362006f6f0 2024-03-16T20:46:09.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f36250b3640 1 --2- 172.21.15.122:0/3733242248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f360c003f00 0x7f360c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f36258b4640 1 --2- 172.21.15.122:0/3733242248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f362006f6f0 0x7f360c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f36260b5640 1 --2- 172.21.15.122:0/3733242248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f360c004d10 0x7f360c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f36250b3640 1 -- 172.21.15.122:0/3733242248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f362006f6f0 msgr2=0x7f360c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:09.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f36250b3640 1 --2- 172.21.15.122:0/3733242248 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f362006f6f0 0x7f360c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f36250b3640 1 -- 172.21.15.122:0/3733242248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f360c004d10 msgr2=0x7f360c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:09.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f36250b3640 1 --2- 172.21.15.122:0/3733242248 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f360c004d10 0x7f360c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f36250b3640 1 -- 172.21.15.122:0/3733242248 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360c003530 con 0x7f360c003f00 2024-03-16T20:46:09.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f36250b3640 1 --2- 172.21.15.122:0/3733242248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f360c003f00 0x7f360c0047d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f361c009ae0 tx=0x7f361c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d933f1cc5dc2f9e6 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f36248b2640 1 -- 172.21.15.122:0/3733242248 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c0a5550 con 0x7f360c003f00 2024-03-16T20:46:09.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f36248b2640 1 -- 172.21.15.122:0/3733242248 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361c0a56b0 con 0x7f360c003f00 2024-03-16T20:46:09.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f36248b2640 1 -- 172.21.15.122:0/3733242248 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361c0a5a60 con 0x7f360c003f00 2024-03-16T20:46:09.079 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f3627b18640 1 -- 172.21.15.122:0/3733242248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f360c003f00 msgr2=0x7f360c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f3627b18640 1 --2- 172.21.15.122:0/3733242248 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f360c003f00 0x7f360c0047d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f361c009ae0 tx=0x7f361c01ca70 comp rx=0 tx=0).stop 2024-03-16T20:46:09.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f36260b5640 1 -- 172.21.15.122:0/3733242248 reap_dead start 2024-03-16T20:46:09.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f3627b18640 1 -- 172.21.15.122:0/3733242248 shutdown_connections 2024-03-16T20:46:09.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f3627b18640 1 -- 172.21.15.122:0/3733242248 >> 172.21.15.122:0/3733242248 conn(0x7f3620070d30 msgr2=0x7f3620071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:09.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f3627b18640 1 -- 172.21.15.122:0/3733242248 shutdown_connections 2024-03-16T20:46:09.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.066+0000 7f3627b18640 1 -- 172.21.15.122:0/3733242248 wait complete. 2024-03-16T20:46:09.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f3627b18640 1 Processor -- start 2024-03-16T20:46:09.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f3627b18640 1 -- start start 2024-03-16T20:46:09.080 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f3627b18640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f360c003f00 0x7f36201420d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f3627b18640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3620143850 0x7f36201dfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f3627b18640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3620143cd0 0x7f36201e03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f3627b18640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36201447a0 con 0x7f3620143cd0 2024-03-16T20:46:09.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f3627b18640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36201d1050 con 0x7f3620143850 2024-03-16T20:46:09.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f3627b18640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3620144c80 con 0x7f360c003f00 2024-03-16T20:46:09.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f36258b4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f360c003f00 0x7f36201420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f36250b3640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3620143850 0x7f36201dfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.081 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f36258b4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f360c003f00 0x7f36201420d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:51320/0 (socket says 172.21.15.122:51320) 2024-03-16T20:46:09.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f36258b4640 1 -- 172.21.15.122:0/2787584767 learned_addr learned my addr 172.21.15.122:0/2787584767 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:09.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f36260b5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3620143cd0 0x7f36201e03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f36260b5640 1 -- 172.21.15.122:0/2787584767 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f360c003f00 msgr2=0x7f36201420d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f36260b5640 1 --2- 172.21.15.122:0/2787584767 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f360c003f00 0x7f36201420d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f36260b5640 1 -- 172.21.15.122:0/2787584767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3620143850 msgr2=0x7f36201dfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f36260b5640 1 --2- 172.21.15.122:0/2787584767 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3620143850 0x7f36201dfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f36260b5640 1 -- 172.21.15.122:0/2787584767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36201d12a0 con 0x7f3620143cd0 2024-03-16T20:46:09.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f36260b5640 1 --2- 172.21.15.122:0/2787584767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3620143cd0 0x7f36201e03f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f362004f900 tx=0x7f3618011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.082 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f36167fc640 1 -- 172.21.15.122:0/2787584767 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361800f8a0 con 0x7f3620143cd0 2024-03-16T20:46:09.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f36167fc640 1 -- 172.21.15.122:0/2787584767 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361800fa40 con 0x7f3620143cd0 2024-03-16T20:46:09.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f36167fc640 1 -- 172.21.15.122:0/2787584767 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361800fdf0 con 0x7f3620143cd0 2024-03-16T20:46:09.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f3627b18640 1 -- 172.21.15.122:0/2787584767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36201449a0 con 0x7f3620143cd0 2024-03-16T20:46:09.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f3627b18640 1 -- 172.21.15.122:0/2787584767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3620143050 con 0x7f3620143cd0 2024-03-16T20:46:09.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f3627b18640 1 -- 172.21.15.122:0/2787584767 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3620147300 con 0x7f3620143cd0 2024-03-16T20:46:09.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.070+0000 7f36167fc640 1 -- 172.21.15.122:0/2787584767 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f361801f460 con 0x7f3620143cd0 2024-03-16T20:46:09.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.074+0000 7f36167fc640 1 --2- 172.21.15.122:0/2787584767 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3604066e40 0x7f3604069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.083 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.074+0000 7f36167fc640 1 -- 172.21.15.122:0/2787584767 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3618087520 con 0x7f3620143cd0 2024-03-16T20:46:09.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.074+0000 7f36167fc640 1 --2- 172.21.15.122:0/2787584767 >> [v2:172.21.15.122:6808/4193825355,v1:172.21.15.122:6809/4193825355] conn(0x7f360406cd00 0x7f360406f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.074+0000 7f36167fc640 1 -- 172.21.15.122:0/2787584767 --> [v2:172.21.15.122:6808/4193825355,v1:172.21.15.122:6809/4193825355] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f360406f7d0 con 0x7f360406cd00 2024-03-16T20:46:09.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.074+0000 7f36167fc640 1 -- 172.21.15.122:0/2787584767 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3618089be0 con 0x7f3620143cd0 2024-03-16T20:46:09.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.074+0000 7f36258b4640 1 --2- 172.21.15.122:0/2787584767 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3604066e40 0x7f3604069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.074+0000 7f36250b3640 1 --2- 172.21.15.122:0/2787584767 >> [v2:172.21.15.122:6808/4193825355,v1:172.21.15.122:6809/4193825355] conn(0x7f360406cd00 0x7f360406f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.074+0000 7f36250b3640 1 --2- 172.21.15.122:0/2787584767 >> [v2:172.21.15.122:6808/4193825355,v1:172.21.15.122:6809/4193825355] conn(0x7f360406cd00 0x7f360406f120 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.084 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.074+0000 7f36258b4640 1 --2- 172.21.15.122:0/2787584767 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3604066e40 0x7f3604069300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3610006c10 tx=0x7f3610008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.085 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.074+0000 7f36167fc640 1 -- 172.21.15.122:0/2787584767 <== osd.0 v2:172.21.15.122:6808/4193825355 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f360406f7d0 con 0x7f360406cd00 2024-03-16T20:46:09.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.086+0000 7f3627b18640 1 -- 172.21.15.122:0/2787584767 --> [v2:172.21.15.122:6808/4193825355,v1:172.21.15.122:6809/4193825355] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f36200b94d0 con 0x7f360406cd00 2024-03-16T20:46:09.093 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.086+0000 7f36167fc640 1 -- 172.21.15.122:0/2787584767 <== osd.0 v2:172.21.15.122:6808/4193825355 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f361c00f080 con 0x7f360406cd00 2024-03-16T20:46:09.093 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-16T20:46:09.093 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T20:46:09.094 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-16T20:46:09.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.086+0000 7f3627b18640 1 -- 172.21.15.122:0/2787584767 >> [v2:172.21.15.122:6808/4193825355,v1:172.21.15.122:6809/4193825355] conn(0x7f360406cd00 msgr2=0x7f360406f120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.086+0000 7f3627b18640 1 --2- 172.21.15.122:0/2787584767 >> [v2:172.21.15.122:6808/4193825355,v1:172.21.15.122:6809/4193825355] conn(0x7f360406cd00 0x7f360406f120 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.086+0000 7f3627b18640 1 -- 172.21.15.122:0/2787584767 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3604066e40 msgr2=0x7f3604069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.086+0000 7f3627b18640 1 --2- 172.21.15.122:0/2787584767 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3604066e40 0x7f3604069300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3610006c10 tx=0x7f3610008040 comp rx=0 tx=0).stop 2024-03-16T20:46:09.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.086+0000 7f3627b18640 1 -- 172.21.15.122:0/2787584767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3620143cd0 msgr2=0x7f36201e03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.086+0000 7f3627b18640 1 --2- 172.21.15.122:0/2787584767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3620143cd0 0x7f36201e03f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f362004f900 tx=0x7f3618011c20 comp rx=0 tx=0).stop 2024-03-16T20:46:09.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.086+0000 7f36260b5640 1 -- 172.21.15.122:0/2787584767 reap_dead start 2024-03-16T20:46:09.095 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.086+0000 7f3627b18640 1 -- 172.21.15.122:0/2787584767 shutdown_connections 2024-03-16T20:46:09.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.086+0000 7f3627b18640 1 -- 172.21.15.122:0/2787584767 >> 172.21.15.122:0/2787584767 conn(0x7f3620070d30 msgr2=0x7f3620071fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:09.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.086+0000 7f3627b18640 1 -- 172.21.15.122:0/2787584767 shutdown_connections 2024-03-16T20:46:09.096 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.086+0000 7f3627b18640 1 -- 172.21.15.122:0/2787584767 wait complete. 2024-03-16T20:46:09.108 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-03-16T20:46:09.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00e2e2e640 1 Processor -- start 2024-03-16T20:46:09.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00e2e2e640 1 -- start start 2024-03-16T20:46:09.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00e2e2e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00dc09ae30 0x7f00dc0af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00e2e2e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00dc0b38e0 con 0x7f00dc0b6f00 2024-03-16T20:46:09.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00e2e2e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00dc09b210 con 0x7f00dc0b4020 2024-03-16T20:46:09.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00e2e2e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc0b4fb0 con 0x7f00dc09ae30 2024-03-16T20:46:09.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00e13cb640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f00dc0b6f00 0x7f00dc14e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:42130/0 (socket says 172.21.15.122:42130) 2024-03-16T20:46:09.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00e13cb640 1 -- 172.21.15.122:0/715042711 learned_addr learned my addr 172.21.15.122:0/715042711 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:46:09.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00e0bca640 1 --2- 172.21.15.122:0/715042711 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00dc09ae30 0x7f00dc0af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00db7fe640 1 -- 172.21.15.122:0/715042711 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850709259 0 0) 0x7f00dc0b38e0 con 0x7f00dc0b6f00 2024-03-16T20:46:09.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00db7fe640 1 -- 172.21.15.122:0/715042711 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00cc003530 con 0x7f00dc0b6f00 2024-03-16T20:46:09.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00db7fe640 1 -- 172.21.15.122:0/715042711 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 914527615 0 0) 0x7f00cc003530 con 0x7f00dc0b6f00 2024-03-16T20:46:09.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00db7fe640 1 -- 172.21.15.122:0/715042711 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00dc09ae30 msgr2=0x7f00dc0af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:09.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00db7fe640 1 --2- 172.21.15.122:0/715042711 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00dc09ae30 0x7f00dc0af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00db7fe640 1 -- 172.21.15.122:0/715042711 >> v1:172.21.15.195:6789/0 conn(0x7f00dc0b4020 legacy=0x7f00dc0b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00db7fe640 1 -- 172.21.15.122:0/715042711 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc0b3290 con 0x7f00dc0b6f00 2024-03-16T20:46:09.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00db7fe640 1 -- 172.21.15.122:0/715042711 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f00d4001a00 con 0x7f00dc0b6f00 2024-03-16T20:46:09.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00db7fe640 1 -- 172.21.15.122:0/715042711 >> v1:172.21.15.122:6789/0 conn(0x7f00dc0b6f00 legacy=0x7f00dc14e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00db7fe640 1 --2- 172.21.15.122:0/715042711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00cc0039b0 0x7f00cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.223 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00db7fe640 1 --2- 172.21.15.122:0/715042711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00cc0047d0 0x7f00cc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00db7fe640 1 --2- 172.21.15.122:0/715042711 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00dc09ae30 0x7f00cc0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00db7fe640 1 -- 172.21.15.122:0/715042711 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc0b4fb0 con 0x7f00cc0047d0 2024-03-16T20:46:09.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00db7fe640 1 -- 172.21.15.122:0/715042711 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc00bac0 con 0x7f00cc0039b0 2024-03-16T20:46:09.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00db7fe640 1 -- 172.21.15.122:0/715042711 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc00de90 con 0x7f00dc09ae30 2024-03-16T20:46:09.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00dbfff640 1 --2- 172.21.15.122:0/715042711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00cc0047d0 0x7f00cc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00e0bca640 1 --2- 172.21.15.122:0/715042711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00cc0039b0 0x7f00cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00e13cb640 1 --2- 172.21.15.122:0/715042711 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00dc09ae30 0x7f00cc0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00dbfff640 1 -- 172.21.15.122:0/715042711 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00dc09ae30 msgr2=0x7f00cc0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:09.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00dbfff640 1 --2- 172.21.15.122:0/715042711 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00dc09ae30 0x7f00cc0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00dbfff640 1 -- 172.21.15.122:0/715042711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00cc0039b0 msgr2=0x7f00cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:09.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00dbfff640 1 --2- 172.21.15.122:0/715042711 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00cc0039b0 0x7f00cc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.214+0000 7f00dbfff640 1 -- 172.21.15.122:0/715042711 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc003530 con 0x7f00cc0047d0 2024-03-16T20:46:09.228 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00dbfff640 1 --2- 172.21.15.122:0/715042711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00cc0047d0 0x7f00cc004c50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f00c8001750 tx=0x7f00c801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232936b940eb23d7 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00db7fe640 1 -- 172.21.15.122:0/715042711 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c80052d0 con 0x7f00cc0047d0 2024-03-16T20:46:09.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00db7fe640 1 -- 172.21.15.122:0/715042711 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c80a7050 con 0x7f00cc0047d0 2024-03-16T20:46:09.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00db7fe640 1 -- 172.21.15.122:0/715042711 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c8003600 con 0x7f00cc0047d0 2024-03-16T20:46:09.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e2e2e640 1 -- 172.21.15.122:0/715042711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00cc0047d0 msgr2=0x7f00cc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e2e2e640 1 --2- 172.21.15.122:0/715042711 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00cc0047d0 0x7f00cc004c50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f00c8001750 tx=0x7f00c801c3b0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e13cb640 1 -- 172.21.15.122:0/715042711 reap_dead start 2024-03-16T20:46:09.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e2e2e640 1 -- 172.21.15.122:0/715042711 shutdown_connections 2024-03-16T20:46:09.229 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e2e2e640 1 -- 172.21.15.122:0/715042711 >> 172.21.15.122:0/715042711 conn(0x7f00dc0b8d10 msgr2=0x7f00dc09a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e2e2e640 1 -- 172.21.15.122:0/715042711 shutdown_connections 2024-03-16T20:46:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e2e2e640 1 -- 172.21.15.122:0/715042711 wait complete. 2024-03-16T20:46:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e2e2e640 1 Processor -- start 2024-03-16T20:46:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e2e2e640 1 -- start start 2024-03-16T20:46:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e2e2e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00cc0047d0 0x7f00dc141260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e2e2e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00dc144fa0 0x7f00dc071580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e2e2e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00dc147a80 0x7f00dc147f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.230 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e2e2e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc1460b0 con 0x7f00dc147a80 2024-03-16T20:46:09.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e2e2e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc1df400 con 0x7f00cc0047d0 2024-03-16T20:46:09.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e2e2e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00dc1dea60 con 0x7f00dc144fa0 2024-03-16T20:46:09.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e13cb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00dc147a80 0x7f00dc147f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e13cb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00dc147a80 0x7f00dc147f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54574/0 (socket says 172.21.15.122:54574) 2024-03-16T20:46:09.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e13cb640 1 -- 172.21.15.122:0/966132538 learned_addr learned my addr 172.21.15.122:0/966132538 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:09.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e0bca640 1 --2- 172.21.15.122:0/966132538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00cc0047d0 0x7f00dc141260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00dbfff640 1 --2- 172.21.15.122:0/966132538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00dc144fa0 0x7f00dc071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e13cb640 1 -- 172.21.15.122:0/966132538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00dc144fa0 msgr2=0x7f00dc071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e13cb640 1 --2- 172.21.15.122:0/966132538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00dc144fa0 0x7f00dc071580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e13cb640 1 -- 172.21.15.122:0/966132538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00cc0047d0 msgr2=0x7f00dc141260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e13cb640 1 --2- 172.21.15.122:0/966132538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00cc0047d0 0x7f00dc141260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e13cb640 1 -- 172.21.15.122:0/966132538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc1ded40 con 0x7f00dc147a80 2024-03-16T20:46:09.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e0bca640 1 --2- 172.21.15.122:0/966132538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00cc0047d0 0x7f00dc141260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:46:09.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00e13cb640 1 --2- 172.21.15.122:0/966132538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00dc147a80 0x7f00dc147f00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f00dc06e0c0 tx=0x7f00d400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.218+0000 7f00d97fa640 1 -- 172.21.15.122:0/966132538 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d400ec10 con 0x7f00dc147a80 2024-03-16T20:46:09.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.222+0000 7f00d97fa640 1 -- 172.21.15.122:0/966132538 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00d400ed70 con 0x7f00dc147a80 2024-03-16T20:46:09.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.222+0000 7f00e2e2e640 1 -- 172.21.15.122:0/966132538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00dc1df020 con 0x7f00dc147a80 2024-03-16T20:46:09.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.222+0000 7f00d97fa640 1 -- 172.21.15.122:0/966132538 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d400eee0 con 0x7f00dc147a80 2024-03-16T20:46:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.222+0000 7f00e2e2e640 1 -- 172.21.15.122:0/966132538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00dc148440 con 0x7f00dc147a80 2024-03-16T20:46:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.222+0000 7f00e2e2e640 1 -- 172.21.15.122:0/966132538 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00dc1df540 con 0x7f00dc147a80 2024-03-16T20:46:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.222+0000 7f00d97fa640 1 -- 172.21.15.122:0/966132538 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f00d4013660 con 0x7f00dc147a80 2024-03-16T20:46:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.222+0000 7f00d97fa640 1 --2- 172.21.15.122:0/966132538 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f00c0066e40 0x7f00c0069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.222+0000 7f00d97fa640 1 -- 172.21.15.122:0/966132538 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f00d4084b80 con 0x7f00dc147a80 2024-03-16T20:46:09.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.222+0000 7f00e0bca640 1 --2- 172.21.15.122:0/966132538 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f00c0066e40 0x7f00c0069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.222+0000 7f00e0bca640 1 --2- 172.21.15.122:0/966132538 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f00c0066e40 0x7f00c0069300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009af0 tx=0x7f00d0008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.226+0000 7f00d97fa640 1 -- 172.21.15.122:0/966132538 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f00dc1df540 con 0x7f00dc147a80 2024-03-16T20:46:09.377 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.370+0000 7f00e2e2e640 1 -- 172.21.15.122:0/966132538 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f00dc09c6f0 con 0x7f00c0066e40 2024-03-16T20:46:09.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.374+0000 7f00d97fa640 1 -- 172.21.15.122:0/966132538 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f00dc09c6f0 con 0x7f00c0066e40 2024-03-16T20:46:09.378 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 0 to deep-scrub 2024-03-16T20:46:09.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.374+0000 7f00e2e2e640 1 -- 172.21.15.122:0/966132538 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f00c0066e40 msgr2=0x7f00c0069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.374+0000 7f00e2e2e640 1 --2- 172.21.15.122:0/966132538 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f00c0066e40 0x7f00c0069300 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009af0 tx=0x7f00d0008040 comp rx=0 tx=0).stop 2024-03-16T20:46:09.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.374+0000 7f00e2e2e640 1 -- 172.21.15.122:0/966132538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00dc147a80 msgr2=0x7f00dc147f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.374+0000 7f00e2e2e640 1 --2- 172.21.15.122:0/966132538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00dc147a80 0x7f00dc147f00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f00dc06e0c0 tx=0x7f00d400e530 comp rx=0 tx=0).stop 2024-03-16T20:46:09.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.374+0000 7f00e2e2e640 1 -- 172.21.15.122:0/966132538 shutdown_connections 2024-03-16T20:46:09.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.374+0000 7f00e2e2e640 1 --2- 172.21.15.122:0/966132538 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f00c0066e40 0x7f00c0069300 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.374+0000 7f00e2e2e640 1 --2- 172.21.15.122:0/966132538 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00dc147a80 0x7f00dc147f00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.374+0000 7f00e2e2e640 1 --2- 172.21.15.122:0/966132538 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f00dc144fa0 0x7f00dc071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.374+0000 7f00e2e2e640 1 --2- 172.21.15.122:0/966132538 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f00cc0047d0 0x7f00dc141260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.374+0000 7f00e2e2e640 1 -- 172.21.15.122:0/966132538 >> 172.21.15.122:0/966132538 conn(0x7f00dc0b8d10 msgr2=0x7f00dc09ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:09.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.374+0000 7f00e2e2e640 1 -- 172.21.15.122:0/966132538 shutdown_connections 2024-03-16T20:46:09.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.374+0000 7f00e2e2e640 1 -- 172.21.15.122:0/966132538 wait complete. 2024-03-16T20:46:09.396 INFO:tasks.ceph:Scrubbing osd.1 2024-03-16T20:46:09.396 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-03-16T20:46:09.518 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb85f987640 1 Processor -- start 2024-03-16T20:46:09.519 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb85f987640 1 -- start start 2024-03-16T20:46:09.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb85f987640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb8580b82b0 0x7fb8580b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb85f987640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb8581422e0 0x7fb858142770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.520 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb85f987640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8580b3fa0 con 0x7fb85809ae30 2024-03-16T20:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb85f987640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8580b50a0 con 0x7fb8580b82b0 2024-03-16T20:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb85f987640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb858142cb0 con 0x7fb8581422e0 2024-03-16T20:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb85d723640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fb85809ae30 0x7fb8580b7ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:42142/0 (socket says 172.21.15.122:42142) 2024-03-16T20:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb85cf22640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fb8580b82b0 0x7fb8580b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb85d723640 1 -- 172.21.15.122:0/2592781384 learned_addr learned my addr 172.21.15.122:0/2592781384 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb85df24640 1 --2- 172.21.15.122:0/2592781384 >> v2:172.21.15.195:3300/0 conn(0x7fb8581422e0 0x7fb858142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.521 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb85cf22640 1 -- 172.21.15.122:0/2592781384 >> v2:172.21.15.195:3300/0 conn(0x7fb8581422e0 msgr2=0x7fb858142770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:09.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb85cf22640 1 --2- 172.21.15.122:0/2592781384 >> v2:172.21.15.195:3300/0 conn(0x7fb8581422e0 0x7fb858142770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb85cf22640 1 -- 172.21.15.122:0/2592781384 >> v1:172.21.15.122:6789/0 conn(0x7fb85809ae30 legacy=0x7fb8580b7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb85cf22640 1 -- 172.21.15.122:0/2592781384 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85809b210 con 0x7fb8580b82b0 2024-03-16T20:46:09.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb84ffff640 1 -- 172.21.15.122:0/2592781384 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2150819732 0 0) 0x7fb8580b3fa0 con 0x7fb85809ae30 2024-03-16T20:46:09.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb85cf22640 1 --2- 172.21.15.122:0/2592781384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8580b82b0 0x7fb8580b4430 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb85809b400 tx=0x7fb84801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efc7bd2a859256e server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb84ffff640 1 -- 172.21.15.122:0/2592781384 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8480ab660 con 0x7fb8580b82b0 2024-03-16T20:46:09.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb84ffff640 1 -- 172.21.15.122:0/2592781384 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8480ab7c0 con 0x7fb8580b82b0 2024-03-16T20:46:09.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.514+0000 7fb84ffff640 1 -- 172.21.15.122:0/2592781384 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8480abb50 con 0x7fb8580b82b0 2024-03-16T20:46:09.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 -- 172.21.15.122:0/2592781384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8580b82b0 msgr2=0x7fb8580b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.524 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 --2- 172.21.15.122:0/2592781384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8580b82b0 0x7fb8580b4430 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fb85809b400 tx=0x7fb84801c460 comp rx=0 tx=0).stop 2024-03-16T20:46:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 -- 172.21.15.122:0/2592781384 shutdown_connections 2024-03-16T20:46:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 --2- 172.21.15.122:0/2592781384 >> v2:172.21.15.195:3300/0 conn(0x7fb8581422e0 0x7fb858142770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 --2- 172.21.15.122:0/2592781384 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8580b82b0 0x7fb8580b4430 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 -- 172.21.15.122:0/2592781384 >> 172.21.15.122:0/2592781384 conn(0x7fb8580b8d10 msgr2=0x7fb85809a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 -- 172.21.15.122:0/2592781384 shutdown_connections 2024-03-16T20:46:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 -- 172.21.15.122:0/2592781384 wait complete. 2024-03-16T20:46:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 Processor -- start 2024-03-16T20:46:09.525 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 -- start start 2024-03-16T20:46:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb85809ae30 0x7fb858144470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8580b82b0 0x7fb85813e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8581422e0 0x7fb85813ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8580b08e0 con 0x7fb8580b82b0 2024-03-16T20:46:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8581e1820 con 0x7fb85809ae30 2024-03-16T20:46:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8580b1260 con 0x7fb8581422e0 2024-03-16T20:46:09.529 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85cf22640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8580b82b0 0x7fb85813e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85cf22640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8580b82b0 0x7fb85813e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54588/0 (socket says 172.21.15.122:54588) 2024-03-16T20:46:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85cf22640 1 -- 172.21.15.122:0/1657402553 learned_addr learned my addr 172.21.15.122:0/1657402553 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85cf22640 1 -- 172.21.15.122:0/1657402553 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8581422e0 msgr2=0x7fb85813ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85cf22640 1 --2- 172.21.15.122:0/1657402553 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb8581422e0 0x7fb85813ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85d723640 1 --2- 172.21.15.122:0/1657402553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb85809ae30 0x7fb858144470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85cf22640 1 -- 172.21.15.122:0/1657402553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb85809ae30 msgr2=0x7fb858144470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.530 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85cf22640 1 --2- 172.21.15.122:0/1657402553 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb85809ae30 0x7fb858144470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85cf22640 1 -- 172.21.15.122:0/1657402553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb848009420 con 0x7fb8580b82b0 2024-03-16T20:46:09.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85cf22640 1 --2- 172.21.15.122:0/1657402553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8580b82b0 0x7fb85813e740 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb848019070 tx=0x7fb84801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb84dffb640 1 -- 172.21.15.122:0/1657402553 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8480b5810 con 0x7fb8580b82b0 2024-03-16T20:46:09.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb84dffb640 1 -- 172.21.15.122:0/1657402553 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8480b5990 con 0x7fb8580b82b0 2024-03-16T20:46:09.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 -- 172.21.15.122:0/1657402553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8581e11d0 con 0x7fb8580b82b0 2024-03-16T20:46:09.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb84dffb640 1 -- 172.21.15.122:0/1657402553 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8480b5d40 con 0x7fb8580b82b0 2024-03-16T20:46:09.531 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 -- 172.21.15.122:0/1657402553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8581449b0 con 0x7fb8580b82b0 2024-03-16T20:46:09.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.518+0000 7fb85f987640 1 -- 172.21.15.122:0/1657402553 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb8580b6720 con 0x7fb8580b82b0 2024-03-16T20:46:09.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.522+0000 7fb84dffb640 1 -- 172.21.15.122:0/1657402553 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fb8480a9070 con 0x7fb8580b82b0 2024-03-16T20:46:09.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.522+0000 7fb84dffb640 1 --2- 172.21.15.122:0/1657402553 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb83c066e40 0x7fb83c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.522+0000 7fb84dffb640 1 -- 172.21.15.122:0/1657402553 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb84811f270 con 0x7fb8580b82b0 2024-03-16T20:46:09.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.522+0000 7fb84dffb640 1 --2- 172.21.15.122:0/1657402553 >> [v2:172.21.15.122:6800/685793600,v1:172.21.15.122:6801/685793600] conn(0x7fb83c06cd00 0x7fb83c06f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.522+0000 7fb84dffb640 1 -- 172.21.15.122:0/1657402553 --> [v2:172.21.15.122:6800/685793600,v1:172.21.15.122:6801/685793600] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb83c06f7d0 con 0x7fb83c06cd00 2024-03-16T20:46:09.532 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.522+0000 7fb84dffb640 1 -- 172.21.15.122:0/1657402553 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb848121930 con 0x7fb8580b82b0 2024-03-16T20:46:09.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.522+0000 7fb85d723640 1 --2- 172.21.15.122:0/1657402553 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb83c066e40 0x7fb83c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.522+0000 7fb85df24640 1 --2- 172.21.15.122:0/1657402553 >> [v2:172.21.15.122:6800/685793600,v1:172.21.15.122:6801/685793600] conn(0x7fb83c06cd00 0x7fb83c06f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.522+0000 7fb85df24640 1 --2- 172.21.15.122:0/1657402553 >> [v2:172.21.15.122:6800/685793600,v1:172.21.15.122:6801/685793600] conn(0x7fb83c06cd00 0x7fb83c06f120 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.522+0000 7fb85d723640 1 --2- 172.21.15.122:0/1657402553 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb83c066e40 0x7fb83c069300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb8500012f0 tx=0x7fb850009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.533 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.522+0000 7fb84dffb640 1 -- 172.21.15.122:0/1657402553 <== osd.1 v2:172.21.15.122:6800/685793600 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb83c06f7d0 con 0x7fb83c06cd00 2024-03-16T20:46:09.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.534+0000 7fb85f987640 1 -- 172.21.15.122:0/1657402553 --> [v2:172.21.15.122:6800/685793600,v1:172.21.15.122:6801/685793600] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb858098620 con 0x7fb83c06cd00 2024-03-16T20:46:09.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.534+0000 7fb84dffb640 1 -- 172.21.15.122:0/1657402553 <== osd.1 v2:172.21.15.122:6800/685793600 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb858098620 con 0x7fb83c06cd00 2024-03-16T20:46:09.542 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-16T20:46:09.543 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T20:46:09.543 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-16T20:46:09.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.538+0000 7fb85f987640 1 -- 172.21.15.122:0/1657402553 >> [v2:172.21.15.122:6800/685793600,v1:172.21.15.122:6801/685793600] conn(0x7fb83c06cd00 msgr2=0x7fb83c06f120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.538+0000 7fb85f987640 1 --2- 172.21.15.122:0/1657402553 >> [v2:172.21.15.122:6800/685793600,v1:172.21.15.122:6801/685793600] conn(0x7fb83c06cd00 0x7fb83c06f120 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.538+0000 7fb85f987640 1 -- 172.21.15.122:0/1657402553 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb83c066e40 msgr2=0x7fb83c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.538+0000 7fb85f987640 1 --2- 172.21.15.122:0/1657402553 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb83c066e40 0x7fb83c069300 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb8500012f0 tx=0x7fb850009040 comp rx=0 tx=0).stop 2024-03-16T20:46:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.538+0000 7fb85f987640 1 -- 172.21.15.122:0/1657402553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8580b82b0 msgr2=0x7fb85813e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.538+0000 7fb85f987640 1 --2- 172.21.15.122:0/1657402553 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb8580b82b0 0x7fb85813e740 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb848019070 tx=0x7fb84801aca0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.538+0000 7fb85df24640 1 -- 172.21.15.122:0/1657402553 reap_dead start 2024-03-16T20:46:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.538+0000 7fb85f987640 1 -- 172.21.15.122:0/1657402553 shutdown_connections 2024-03-16T20:46:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.538+0000 7fb85f987640 1 -- 172.21.15.122:0/1657402553 >> 172.21.15.122:0/1657402553 conn(0x7fb8580b8d10 msgr2=0x7fb858145100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:09.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.538+0000 7fb85f987640 1 -- 172.21.15.122:0/1657402553 shutdown_connections 2024-03-16T20:46:09.546 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.538+0000 7fb85f987640 1 -- 172.21.15.122:0/1657402553 wait complete. 2024-03-16T20:46:09.558 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-03-16T20:46:09.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d985c640 1 Processor -- start 2024-03-16T20:46:09.662 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d985c640 1 -- start start 2024-03-16T20:46:09.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d985c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1d4148480 0x7fc1d40b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d985c640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d40b6740 con 0x7fc1d40b7540 2024-03-16T20:46:09.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d985c640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d40b7920 con 0x7fc1d40b7d40 2024-03-16T20:46:09.665 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d985c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d40b23d0 con 0x7fc1d4148480 2024-03-16T20:46:09.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d2ffd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fc1d40b7540 0x7fc1d413ef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:42154/0 (socket says 172.21.15.122:42154) 2024-03-16T20:46:09.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d2ffd640 1 -- 172.21.15.122:0/2634332365 learned_addr learned my addr 172.21.15.122:0/2634332365 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:46:09.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d37fe640 1 --2- 172.21.15.122:0/2634332365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1d4148480 0x7fc1d40b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d1ffb640 1 -- 172.21.15.122:0/2634332365 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926526457 0 0) 0x7fc1d40b6740 con 0x7fc1d40b7540 2024-03-16T20:46:09.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d1ffb640 1 -- 172.21.15.122:0/2634332365 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1bc003530 con 0x7fc1d40b7540 2024-03-16T20:46:09.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d1ffb640 1 -- 172.21.15.122:0/2634332365 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1850019063 0 0) 0x7fc1bc003530 con 0x7fc1d40b7540 2024-03-16T20:46:09.666 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d1ffb640 1 -- 172.21.15.122:0/2634332365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1d4148480 msgr2=0x7fc1d40b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:09.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d1ffb640 1 --2- 172.21.15.122:0/2634332365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1d4148480 0x7fc1d40b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d1ffb640 1 -- 172.21.15.122:0/2634332365 >> v1:172.21.15.195:6789/0 conn(0x7fc1d40b7d40 legacy=0x7fc1d414d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d1ffb640 1 -- 172.21.15.122:0/2634332365 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d40b31b0 con 0x7fc1d40b7540 2024-03-16T20:46:09.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d1ffb640 1 -- 172.21.15.122:0/2634332365 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7fc1c4001680 con 0x7fc1d40b7540 2024-03-16T20:46:09.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d1ffb640 1 -- 172.21.15.122:0/2634332365 >> v1:172.21.15.122:6789/0 conn(0x7fc1d40b7540 legacy=0x7fc1d413ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d1ffb640 1 --2- 172.21.15.122:0/2634332365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc0039b0 0x7fc1bc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d1ffb640 1 --2- 172.21.15.122:0/2634332365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1d4148480 0x7fc1bc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.667 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d1ffb640 1 --2- 172.21.15.122:0/2634332365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1bc004d10 0x7fc1bc005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d1ffb640 1 -- 172.21.15.122:0/2634332365 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d40b23d0 con 0x7fc1bc0039b0 2024-03-16T20:46:09.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d1ffb640 1 -- 172.21.15.122:0/2634332365 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc00bb50 con 0x7fc1bc004d10 2024-03-16T20:46:09.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d1ffb640 1 -- 172.21.15.122:0/2634332365 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc00bd10 con 0x7fc1d4148480 2024-03-16T20:46:09.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d2ffd640 1 --2- 172.21.15.122:0/2634332365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc0039b0 0x7fc1bc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d27fc640 1 --2- 172.21.15.122:0/2634332365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1d4148480 0x7fc1bc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d37fe640 1 --2- 172.21.15.122:0/2634332365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1bc004d10 0x7fc1bc005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d2ffd640 1 -- 172.21.15.122:0/2634332365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1d4148480 msgr2=0x7fc1bc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:09.668 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d2ffd640 1 --2- 172.21.15.122:0/2634332365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1d4148480 0x7fc1bc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d2ffd640 1 -- 172.21.15.122:0/2634332365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1bc004d10 msgr2=0x7fc1bc005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:09.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d2ffd640 1 --2- 172.21.15.122:0/2634332365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1bc004d10 0x7fc1bc005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d2ffd640 1 -- 172.21.15.122:0/2634332365 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc003530 con 0x7fc1bc0039b0 2024-03-16T20:46:09.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d27fc640 1 --2- 172.21.15.122:0/2634332365 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1d4148480 0x7fc1bc0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:09.669 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.658+0000 7fc1d37fe640 1 --2- 172.21.15.122:0/2634332365 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1bc004d10 0x7fc1bc005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:09.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d2ffd640 1 --2- 172.21.15.122:0/2634332365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc0039b0 0x7fc1bc003dd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400c110 tx=0x7fc1c401cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2b3350ddc219b2 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d1ffb640 1 -- 172.21.15.122:0/2634332365 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c4005ae0 con 0x7fc1bc0039b0 2024-03-16T20:46:09.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d1ffb640 1 -- 172.21.15.122:0/2634332365 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c4005c60 con 0x7fc1bc0039b0 2024-03-16T20:46:09.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d1ffb640 1 -- 172.21.15.122:0/2634332365 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c40a5810 con 0x7fc1bc0039b0 2024-03-16T20:46:09.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d985c640 1 -- 172.21.15.122:0/2634332365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc0039b0 msgr2=0x7fc1bc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d985c640 1 --2- 172.21.15.122:0/2634332365 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc0039b0 0x7fc1bc003dd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc1c400c110 tx=0x7fc1c401cb50 comp rx=0 tx=0).stop 2024-03-16T20:46:09.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d37fe640 1 -- 172.21.15.122:0/2634332365 reap_dead start 2024-03-16T20:46:09.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d985c640 1 -- 172.21.15.122:0/2634332365 shutdown_connections 2024-03-16T20:46:09.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d985c640 1 -- 172.21.15.122:0/2634332365 >> 172.21.15.122:0/2634332365 conn(0x7fc1d406fc00 msgr2=0x7fc1d4070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:09.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d985c640 1 -- 172.21.15.122:0/2634332365 shutdown_connections 2024-03-16T20:46:09.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d985c640 1 -- 172.21.15.122:0/2634332365 wait complete. 2024-03-16T20:46:09.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d985c640 1 Processor -- start 2024-03-16T20:46:09.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d985c640 1 -- start start 2024-03-16T20:46:09.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d985c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc0039b0 0x7fc1d4135e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d985c640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1d41dd9b0 0x7fc1d41dde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d985c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1d4133890 0x7fc1d4133d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d985c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d41de370 con 0x7fc1bc0039b0 2024-03-16T20:46:09.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d985c640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d4143d80 con 0x7fc1d41dd9b0 2024-03-16T20:46:09.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d985c640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d4144040 con 0x7fc1d4133890 2024-03-16T20:46:09.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc0039b0 0x7fc1d4135e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d2ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc0039b0 0x7fc1d4135e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54602/0 (socket says 172.21.15.122:54602) 2024-03-16T20:46:09.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d2ffd640 1 -- 172.21.15.122:0/1740657664 learned_addr learned my addr 172.21.15.122:0/1740657664 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:09.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d2ffd640 1 -- 172.21.15.122:0/1740657664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1d4133890 msgr2=0x7fc1d4133d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d2ffd640 1 --2- 172.21.15.122:0/1740657664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1d4133890 0x7fc1d4133d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.675 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d2ffd640 1 -- 172.21.15.122:0/1740657664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1d41dd9b0 msgr2=0x7fc1d41dde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d2ffd640 1 --2- 172.21.15.122:0/1740657664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1d41dd9b0 0x7fc1d41dde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d2ffd640 1 -- 172.21.15.122:0/1740657664 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d4137050 con 0x7fc1bc0039b0 2024-03-16T20:46:09.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d2ffd640 1 --2- 172.21.15.122:0/1740657664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc0039b0 0x7fc1d4135e00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4000c10 tx=0x7fc1bc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1bbfff640 1 -- 172.21.15.122:0/1740657664 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c40b4070 con 0x7fc1bc0039b0 2024-03-16T20:46:09.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1bbfff640 1 -- 172.21.15.122:0/1740657664 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c40a5810 con 0x7fc1bc0039b0 2024-03-16T20:46:09.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d985c640 1 -- 172.21.15.122:0/1740657664 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d4137330 con 0x7fc1bc0039b0 2024-03-16T20:46:09.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1bbfff640 1 -- 172.21.15.122:0/1740657664 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c40a5ba0 con 0x7fc1bc0039b0 2024-03-16T20:46:09.676 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.662+0000 7fc1d985c640 1 -- 172.21.15.122:0/1740657664 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d4143a40 con 0x7fc1bc0039b0 2024-03-16T20:46:09.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.666+0000 7fc1d985c640 1 -- 172.21.15.122:0/1740657664 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d413b7d0 con 0x7fc1bc0039b0 2024-03-16T20:46:09.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.666+0000 7fc1bbfff640 1 -- 172.21.15.122:0/1740657664 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fc1c40b7070 con 0x7fc1bc0039b0 2024-03-16T20:46:09.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.666+0000 7fc1bbfff640 1 --2- 172.21.15.122:0/1740657664 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc1c0066e70 0x7fc1c0069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.666+0000 7fc1bbfff640 1 -- 172.21.15.122:0/1740657664 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc1c411d8c0 con 0x7fc1bc0039b0 2024-03-16T20:46:09.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.666+0000 7fc1d27fc640 1 --2- 172.21.15.122:0/1740657664 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc1c0066e70 0x7fc1c0069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.666+0000 7fc1d27fc640 1 --2- 172.21.15.122:0/1740657664 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc1c0066e70 0x7fc1c0069330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8007fd0 tx=0x7fc1c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.678 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.670+0000 7fc1bbfff640 1 -- 172.21.15.122:0/1740657664 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fc1c40ea9b0 con 0x7fc1bc0039b0 2024-03-16T20:46:09.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.814+0000 7fc1d985c640 1 -- 172.21.15.122:0/1740657664 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1d40b6fc0 con 0x7fc1c0066e70 2024-03-16T20:46:09.823 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.818+0000 7fc1bbfff640 1 -- 172.21.15.122:0/1740657664 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc1c800d4f0 con 0x7fc1c0066e70 2024-03-16T20:46:09.823 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 1 to deep-scrub 2024-03-16T20:46:09.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.818+0000 7fc1d985c640 1 -- 172.21.15.122:0/1740657664 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc1c0066e70 msgr2=0x7fc1c0069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.818+0000 7fc1d985c640 1 --2- 172.21.15.122:0/1740657664 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc1c0066e70 0x7fc1c0069330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8007fd0 tx=0x7fc1c8009040 comp rx=0 tx=0).stop 2024-03-16T20:46:09.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.818+0000 7fc1d985c640 1 -- 172.21.15.122:0/1740657664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc0039b0 msgr2=0x7fc1d4135e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.818+0000 7fc1d985c640 1 --2- 172.21.15.122:0/1740657664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc0039b0 0x7fc1d4135e00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc1c4000c10 tx=0x7fc1bc005800 comp rx=0 tx=0).stop 2024-03-16T20:46:09.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.818+0000 7fc1d985c640 1 -- 172.21.15.122:0/1740657664 shutdown_connections 2024-03-16T20:46:09.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.818+0000 7fc1d985c640 1 --2- 172.21.15.122:0/1740657664 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc1c0066e70 0x7fc1c0069330 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.818+0000 7fc1d985c640 1 --2- 172.21.15.122:0/1740657664 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc1d4133890 0x7fc1d4133d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.818+0000 7fc1d985c640 1 --2- 172.21.15.122:0/1740657664 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc1d41dd9b0 0x7fc1d41dde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.826 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.818+0000 7fc1d985c640 1 --2- 172.21.15.122:0/1740657664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc1bc0039b0 0x7fc1d4135e00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.818+0000 7fc1d985c640 1 -- 172.21.15.122:0/1740657664 >> 172.21.15.122:0/1740657664 conn(0x7fc1d406fc00 msgr2=0x7fc1d4138450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:09.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.818+0000 7fc1d985c640 1 -- 172.21.15.122:0/1740657664 shutdown_connections 2024-03-16T20:46:09.827 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.818+0000 7fc1d985c640 1 -- 172.21.15.122:0/1740657664 wait complete. 2024-03-16T20:46:09.840 INFO:tasks.ceph:Scrubbing osd.2 2024-03-16T20:46:09.841 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-03-16T20:46:09.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.950+0000 7f9c2ce95640 1 Processor -- start 2024-03-16T20:46:09.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.950+0000 7f9c2ce95640 1 -- start start 2024-03-16T20:46:09.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2ce95640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f9c2809ae30 0x7f9c280af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2ce95640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c280b4020 0x7f9c280b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2ce95640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9c280b7820 0x7f9c280b6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2ce95640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c280b38e0 con 0x7f9c280b7820 2024-03-16T20:46:09.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2ce95640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c2809b210 con 0x7f9c2809ae30 2024-03-16T20:46:09.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2ce95640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c280b4fb0 con 0x7f9c280b4020 2024-03-16T20:46:09.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c26d9d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9c280b7820 0x7f9c280b6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c26d9d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9c280b7820 0x7f9c280b6d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54616/0 (socket says 172.21.15.122:54616) 2024-03-16T20:46:09.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c26d9d640 1 -- 172.21.15.122:0/1809055134 learned_addr learned my addr 172.21.15.122:0/1809055134 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:09.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2659c640 1 --2- 172.21.15.122:0/1809055134 >> v2:172.21.15.195:3300/0 conn(0x7f9c2809ae30 0x7f9c280af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c25d9b640 1 --2- 172.21.15.122:0/1809055134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c280b4020 0x7f9c280b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c26d9d640 1 -- 172.21.15.122:0/1809055134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c280b4020 msgr2=0x7f9c280b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:09.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c26d9d640 1 --2- 172.21.15.122:0/1809055134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c280b4020 0x7f9c280b67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c26d9d640 1 -- 172.21.15.122:0/1809055134 >> v2:172.21.15.195:3300/0 conn(0x7f9c2809ae30 msgr2=0x7f9c280af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:09.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c26d9d640 1 --2- 172.21.15.122:0/1809055134 >> v2:172.21.15.195:3300/0 conn(0x7f9c2809ae30 0x7f9c280af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c26d9d640 1 -- 172.21.15.122:0/1809055134 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c280b3290 con 0x7f9c280b7820 2024-03-16T20:46:09.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c25d9b640 1 --2- 172.21.15.122:0/1809055134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c280b4020 0x7f9c280b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:09.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2659c640 1 --2- 172.21.15.122:0/1809055134 >> v2:172.21.15.195:3300/0 conn(0x7f9c2809ae30 0x7f9c280af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:09.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c26d9d640 1 --2- 172.21.15.122:0/1809055134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c280b7820 0x7f9c280b6d30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9c280b9780 tx=0x7f9c1801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d9e6e63e260ebd server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2559a640 1 -- 172.21.15.122:0/1809055134 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c180ab670 con 0x7f9c280b7820 2024-03-16T20:46:09.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2559a640 1 -- 172.21.15.122:0/1809055134 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c180ab7d0 con 0x7f9c280b7820 2024-03-16T20:46:09.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2559a640 1 -- 172.21.15.122:0/1809055134 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c180abb60 con 0x7f9c280b7820 2024-03-16T20:46:09.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2ce95640 1 -- 172.21.15.122:0/1809055134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c280b7820 msgr2=0x7f9c280b6d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2ce95640 1 --2- 172.21.15.122:0/1809055134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c280b7820 0x7f9c280b6d30 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9c280b9780 tx=0x7f9c1801c490 comp rx=0 tx=0).stop 2024-03-16T20:46:09.965 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2ce95640 1 -- 172.21.15.122:0/1809055134 shutdown_connections 2024-03-16T20:46:09.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2ce95640 1 --2- 172.21.15.122:0/1809055134 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c280b7820 0x7f9c280b6d30 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2ce95640 1 --2- 172.21.15.122:0/1809055134 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c280b4020 0x7f9c280b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2ce95640 1 --2- 172.21.15.122:0/1809055134 >> v2:172.21.15.195:3300/0 conn(0x7f9c2809ae30 0x7f9c280af080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2ce95640 1 -- 172.21.15.122:0/1809055134 >> 172.21.15.122:0/1809055134 conn(0x7f9c280b8d10 msgr2=0x7f9c2809a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:09.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2ce95640 1 -- 172.21.15.122:0/1809055134 shutdown_connections 2024-03-16T20:46:09.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.954+0000 7f9c2ce95640 1 -- 172.21.15.122:0/1809055134 wait complete. 2024-03-16T20:46:09.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c2ce95640 1 Processor -- start 2024-03-16T20:46:09.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c2ce95640 1 -- start start 2024-03-16T20:46:09.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c2ce95640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c2809ae30 0x7f9c28143e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c2ce95640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c280b4020 0x7f9c28143560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c2ce95640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c280b7820 0x7f9c280b1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c2ce95640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c280b1780 con 0x7f9c280b4020 2024-03-16T20:46:09.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c2ce95640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2813de20 con 0x7f9c2809ae30 2024-03-16T20:46:09.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c2ce95640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c280b0ad0 con 0x7f9c280b7820 2024-03-16T20:46:09.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c25d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c280b4020 0x7f9c28143560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c25d9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c280b4020 0x7f9c28143560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54622/0 (socket says 172.21.15.122:54622) 2024-03-16T20:46:09.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c25d9b640 1 -- 172.21.15.122:0/2277554974 learned_addr learned my addr 172.21.15.122:0/2277554974 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:09.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c2659c640 1 --2- 172.21.15.122:0/2277554974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c2809ae30 0x7f9c28143e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c26d9d640 1 --2- 172.21.15.122:0/2277554974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c280b7820 0x7f9c280b1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c25d9b640 1 -- 172.21.15.122:0/2277554974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c280b7820 msgr2=0x7f9c280b1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c25d9b640 1 --2- 172.21.15.122:0/2277554974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c280b7820 0x7f9c280b1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c25d9b640 1 -- 172.21.15.122:0/2277554974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c2809ae30 msgr2=0x7f9c28143e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c25d9b640 1 --2- 172.21.15.122:0/2277554974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c2809ae30 0x7f9c28143e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c25d9b640 1 -- 172.21.15.122:0/2277554974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c180094d0 con 0x7f9c280b4020 2024-03-16T20:46:09.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c2659c640 1 --2- 172.21.15.122:0/2277554974 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9c2809ae30 0x7f9c28143e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:09.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c26d9d640 1 --2- 172.21.15.122:0/2277554974 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9c280b7820 0x7f9c280b1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:09.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c25d9b640 1 --2- 172.21.15.122:0/2277554974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c280b4020 0x7f9c28143560 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000c7f0 tx=0x7f9c1000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c1effd640 1 -- 172.21.15.122:0/2277554974 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1000cea0 con 0x7f9c280b4020 2024-03-16T20:46:09.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c1effd640 1 -- 172.21.15.122:0/2277554974 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c10007a30 con 0x7f9c280b4020 2024-03-16T20:46:09.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c2ce95640 1 -- 172.21.15.122:0/2277554974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c280b0db0 con 0x7f9c280b4020 2024-03-16T20:46:09.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c1effd640 1 -- 172.21.15.122:0/2277554974 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c10007dc0 con 0x7f9c280b4020 2024-03-16T20:46:09.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c2ce95640 1 -- 172.21.15.122:0/2277554974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2813e050 con 0x7f9c280b4020 2024-03-16T20:46:09.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c2ce95640 1 -- 172.21.15.122:0/2277554974 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9c2806e0d0 con 0x7f9c280b4020 2024-03-16T20:46:09.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c1effd640 1 -- 172.21.15.122:0/2277554974 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f9c100113f0 con 0x7f9c280b4020 2024-03-16T20:46:09.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c1effd640 1 --2- 172.21.15.122:0/2277554974 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9c08066e40 0x7f9c08069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.958+0000 7f9c1effd640 1 -- 172.21.15.122:0/2277554974 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9c10084d80 con 0x7f9c280b4020 2024-03-16T20:46:09.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.962+0000 7f9c1effd640 1 --2- 172.21.15.122:0/2277554974 >> [v2:172.21.15.122:6819/2377812328,v1:172.21.15.122:6822/2377812328] conn(0x7f9c0806cd00 0x7f9c0806f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:09.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.962+0000 7f9c1effd640 1 -- 172.21.15.122:0/2277554974 --> [v2:172.21.15.122:6819/2377812328,v1:172.21.15.122:6822/2377812328] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9c0806f7d0 con 0x7f9c0806cd00 2024-03-16T20:46:09.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.962+0000 7f9c1effd640 1 -- 172.21.15.122:0/2277554974 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c100851d0 con 0x7f9c280b4020 2024-03-16T20:46:09.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.962+0000 7f9c26d9d640 1 --2- 172.21.15.122:0/2277554974 >> [v2:172.21.15.122:6819/2377812328,v1:172.21.15.122:6822/2377812328] conn(0x7f9c0806cd00 0x7f9c0806f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.962+0000 7f9c2659c640 1 --2- 172.21.15.122:0/2277554974 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9c08066e40 0x7f9c08069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:09.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.962+0000 7f9c26d9d640 1 --2- 172.21.15.122:0/2277554974 >> [v2:172.21.15.122:6819/2377812328,v1:172.21.15.122:6822/2377812328] conn(0x7f9c0806cd00 0x7f9c0806f120 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.962+0000 7f9c2659c640 1 --2- 172.21.15.122:0/2277554974 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9c08066e40 0x7f9c08069300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9c280b2dd0 tx=0x7f9c20007550 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:09.972 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.962+0000 7f9c1effd640 1 -- 172.21.15.122:0/2277554974 <== osd.2 v2:172.21.15.122:6819/2377812328 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f9c0806f7d0 con 0x7f9c0806cd00 2024-03-16T20:46:09.979 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.974+0000 7f9c2ce95640 1 -- 172.21.15.122:0/2277554974 --> [v2:172.21.15.122:6819/2377812328,v1:172.21.15.122:6822/2377812328] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9c28143b60 con 0x7f9c0806cd00 2024-03-16T20:46:09.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.974+0000 7f9c1effd640 1 -- 172.21.15.122:0/2277554974 <== osd.2 v2:172.21.15.122:6819/2377812328 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f9c28143b60 con 0x7f9c0806cd00 2024-03-16T20:46:09.980 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-16T20:46:09.980 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T20:46:09.980 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-16T20:46:09.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.974+0000 7f9c2ce95640 1 -- 172.21.15.122:0/2277554974 >> [v2:172.21.15.122:6819/2377812328,v1:172.21.15.122:6822/2377812328] conn(0x7f9c0806cd00 msgr2=0x7f9c0806f120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.974+0000 7f9c2ce95640 1 --2- 172.21.15.122:0/2277554974 >> [v2:172.21.15.122:6819/2377812328,v1:172.21.15.122:6822/2377812328] conn(0x7f9c0806cd00 0x7f9c0806f120 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.974+0000 7f9c2ce95640 1 -- 172.21.15.122:0/2277554974 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9c08066e40 msgr2=0x7f9c08069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.981 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.974+0000 7f9c2ce95640 1 --2- 172.21.15.122:0/2277554974 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9c08066e40 0x7f9c08069300 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9c280b2dd0 tx=0x7f9c20007550 comp rx=0 tx=0).stop 2024-03-16T20:46:09.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.974+0000 7f9c2ce95640 1 -- 172.21.15.122:0/2277554974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c280b4020 msgr2=0x7f9c28143560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:09.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.974+0000 7f9c2ce95640 1 --2- 172.21.15.122:0/2277554974 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c280b4020 0x7f9c28143560 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000c7f0 tx=0x7f9c1000ccb0 comp rx=0 tx=0).stop 2024-03-16T20:46:09.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.974+0000 7f9c26d9d640 1 -- 172.21.15.122:0/2277554974 reap_dead start 2024-03-16T20:46:09.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.974+0000 7f9c2ce95640 1 -- 172.21.15.122:0/2277554974 shutdown_connections 2024-03-16T20:46:09.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.974+0000 7f9c2ce95640 1 -- 172.21.15.122:0/2277554974 >> 172.21.15.122:0/2277554974 conn(0x7f9c280b8d10 msgr2=0x7f9c28145570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:09.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.974+0000 7f9c2ce95640 1 -- 172.21.15.122:0/2277554974 shutdown_connections 2024-03-16T20:46:09.982 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:09.974+0000 7f9c2ce95640 1 -- 172.21.15.122:0/2277554974 wait complete. 2024-03-16T20:46:09.994 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-03-16T20:46:10.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.090+0000 7f9acd0be640 1 Processor -- start 2024-03-16T20:46:10.098 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.090+0000 7f9acd0be640 1 -- start start 2024-03-16T20:46:10.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.094+0000 7f9acd0be640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ac80b7540 0x7f9ac80b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.094+0000 7f9acd0be640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9ac80b7d40 0x7f9ac813ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.099 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.094+0000 7f9acd0be640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac80b6740 con 0x7f9ac813f460 2024-03-16T20:46:10.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.094+0000 7f9acd0be640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac80b23d0 con 0x7f9ac80b7d40 2024-03-16T20:46:10.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.094+0000 7f9acd0be640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80b7110 con 0x7f9ac80b7540 2024-03-16T20:46:10.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.094+0000 7f9ac659c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9ac80b7d40 0x7f9ac813ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.094+0000 7f9ac659c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f9ac80b7d40 0x7f9ac813ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54632/0 (socket says 172.21.15.122:54632) 2024-03-16T20:46:10.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.094+0000 7f9ac659c640 1 -- 172.21.15.122:0/697878604 learned_addr learned my addr 172.21.15.122:0/697878604 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:10.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.094+0000 7f9ac6d9d640 1 --2- 172.21.15.122:0/697878604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ac80b7540 0x7f9ac80b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.094+0000 7f9ac659c640 1 -- 172.21.15.122:0/697878604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ac80b7540 msgr2=0x7f9ac80b6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:10.100 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.094+0000 7f9ac659c640 1 --2- 172.21.15.122:0/697878604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ac80b7540 0x7f9ac80b6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.094+0000 7f9ac659c640 1 -- 172.21.15.122:0/697878604 >> v1:172.21.15.195:6789/0 conn(0x7f9ac813f460 legacy=0x7f9ac80b3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:10.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.094+0000 7f9ac659c640 1 -- 172.21.15.122:0/697878604 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac80b7920 con 0x7f9ac80b7d40 2024-03-16T20:46:10.101 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.094+0000 7f9ac659c640 1 --2- 172.21.15.122:0/697878604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac80b7d40 0x7f9ac813ef20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0098a0 tx=0x7f9abc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce99b7cee181733f server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:10.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.094+0000 7f9ac5d9b640 1 -- 172.21.15.122:0/697878604 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0aa5f0 con 0x7f9ac80b7d40 2024-03-16T20:46:10.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.094+0000 7f9ac5d9b640 1 -- 172.21.15.122:0/697878604 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc0aa750 con 0x7f9ac80b7d40 2024-03-16T20:46:10.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.094+0000 7f9ac5d9b640 1 -- 172.21.15.122:0/697878604 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc0aaae0 con 0x7f9ac80b7d40 2024-03-16T20:46:10.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 -- 172.21.15.122:0/697878604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac80b7d40 msgr2=0x7f9ac813ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 --2- 172.21.15.122:0/697878604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac80b7d40 0x7f9ac813ef20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9abc0098a0 tx=0x7f9abc01c4d0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 -- 172.21.15.122:0/697878604 shutdown_connections 2024-03-16T20:46:10.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 --2- 172.21.15.122:0/697878604 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac80b7d40 0x7f9ac813ef20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 --2- 172.21.15.122:0/697878604 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ac80b7540 0x7f9ac80b6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 -- 172.21.15.122:0/697878604 >> 172.21.15.122:0/697878604 conn(0x7f9ac806fc00 msgr2=0x7f9ac8070010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:10.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 -- 172.21.15.122:0/697878604 shutdown_connections 2024-03-16T20:46:10.104 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 -- 172.21.15.122:0/697878604 wait complete. 2024-03-16T20:46:10.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 Processor -- start 2024-03-16T20:46:10.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 -- start start 2024-03-16T20:46:10.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ac80b7540 0x7f9ac81dd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ac80b7d40 0x7f9ac813da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac813f460 0x7f9ac81362c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac81386e0 con 0x7f9ac813f460 2024-03-16T20:46:10.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac81388e0 con 0x7f9ac80b7540 2024-03-16T20:46:10.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac8138bc0 con 0x7f9ac80b7d40 2024-03-16T20:46:10.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9ac759e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac813f460 0x7f9ac81362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.107 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9ac759e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac813f460 0x7f9ac81362c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54644/0 (socket says 172.21.15.122:54644) 2024-03-16T20:46:10.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9ac759e640 1 -- 172.21.15.122:0/3972770660 learned_addr learned my addr 172.21.15.122:0/3972770660 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:10.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9ac6d9d640 1 --2- 172.21.15.122:0/3972770660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ac80b7540 0x7f9ac81dd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9ac659c640 1 --2- 172.21.15.122:0/3972770660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ac80b7d40 0x7f9ac813da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9ac759e640 1 -- 172.21.15.122:0/3972770660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ac80b7d40 msgr2=0x7f9ac813da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9ac759e640 1 --2- 172.21.15.122:0/3972770660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ac80b7d40 0x7f9ac813da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9ac759e640 1 -- 172.21.15.122:0/3972770660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ac80b7540 msgr2=0x7f9ac81dd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9ac759e640 1 --2- 172.21.15.122:0/3972770660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ac80b7540 0x7f9ac81dd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9ac759e640 1 -- 172.21.15.122:0/3972770660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9abc0094d0 con 0x7f9ac813f460 2024-03-16T20:46:10.108 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9ac759e640 1 --2- 172.21.15.122:0/3972770660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac813f460 0x7f9ac81362c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000b970 tx=0x7f9ac000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:10.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9aaf7fe640 1 -- 172.21.15.122:0/3972770660 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac000cb40 con 0x7f9ac813f460 2024-03-16T20:46:10.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9aaf7fe640 1 -- 172.21.15.122:0/3972770660 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac000cca0 con 0x7f9ac813f460 2024-03-16T20:46:10.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 -- 172.21.15.122:0/3972770660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac8138ea0 con 0x7f9ac813f460 2024-03-16T20:46:10.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9aaf7fe640 1 -- 172.21.15.122:0/3972770660 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac0014680 con 0x7f9ac813f460 2024-03-16T20:46:10.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 -- 172.21.15.122:0/3972770660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac81e14c0 con 0x7f9ac813f460 2024-03-16T20:46:10.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9acd0be640 1 -- 172.21.15.122:0/3972770660 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac81373f0 con 0x7f9ac813f460 2024-03-16T20:46:10.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.098+0000 7f9aaf7fe640 1 -- 172.21.15.122:0/3972770660 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f9ac001d3b0 con 0x7f9ac813f460 2024-03-16T20:46:10.109 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.102+0000 7f9aaf7fe640 1 --2- 172.21.15.122:0/3972770660 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9ab4066e40 0x7f9ab4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.102+0000 7f9aaf7fe640 1 -- 172.21.15.122:0/3972770660 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9ac0084610 con 0x7f9ac813f460 2024-03-16T20:46:10.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.102+0000 7f9ac6d9d640 1 --2- 172.21.15.122:0/3972770660 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9ab4066e40 0x7f9ab4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.110 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.102+0000 7f9ac6d9d640 1 --2- 172.21.15.122:0/3972770660 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9ab4066e40 0x7f9ab4069300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800b9d0 tx=0x7f9ab8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:10.111 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.106+0000 7f9aaf7fe640 1 -- 172.21.15.122:0/3972770660 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f9ac0051790 con 0x7f9ac813f460 2024-03-16T20:46:10.256 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.250+0000 7f9acd0be640 1 -- 172.21.15.122:0/3972770660 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ac8137110 con 0x7f9ab4066e40 2024-03-16T20:46:10.257 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.250+0000 7f9aaf7fe640 1 -- 172.21.15.122:0/3972770660 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9ac8137110 con 0x7f9ab4066e40 2024-03-16T20:46:10.258 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 2 to deep-scrub 2024-03-16T20:46:10.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.254+0000 7f9acd0be640 1 -- 172.21.15.122:0/3972770660 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9ab4066e40 msgr2=0x7f9ab4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.254+0000 7f9acd0be640 1 --2- 172.21.15.122:0/3972770660 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9ab4066e40 0x7f9ab4069300 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9ab800b9d0 tx=0x7f9ab8008040 comp rx=0 tx=0).stop 2024-03-16T20:46:10.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.254+0000 7f9acd0be640 1 -- 172.21.15.122:0/3972770660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac813f460 msgr2=0x7f9ac81362c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.254+0000 7f9acd0be640 1 --2- 172.21.15.122:0/3972770660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac813f460 0x7f9ac81362c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9ac000b970 tx=0x7f9ac000be30 comp rx=0 tx=0).stop 2024-03-16T20:46:10.260 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.254+0000 7f9acd0be640 1 -- 172.21.15.122:0/3972770660 shutdown_connections 2024-03-16T20:46:10.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.254+0000 7f9acd0be640 1 --2- 172.21.15.122:0/3972770660 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f9ab4066e40 0x7f9ab4069300 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.254+0000 7f9acd0be640 1 --2- 172.21.15.122:0/3972770660 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ac813f460 0x7f9ac81362c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.254+0000 7f9acd0be640 1 --2- 172.21.15.122:0/3972770660 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f9ac80b7d40 0x7f9ac813da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.254+0000 7f9acd0be640 1 --2- 172.21.15.122:0/3972770660 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f9ac80b7540 0x7f9ac81dd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.254+0000 7f9acd0be640 1 -- 172.21.15.122:0/3972770660 >> 172.21.15.122:0/3972770660 conn(0x7f9ac806fc00 msgr2=0x7f9ac806ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:10.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.254+0000 7f9acd0be640 1 -- 172.21.15.122:0/3972770660 shutdown_connections 2024-03-16T20:46:10.261 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.254+0000 7f9acd0be640 1 -- 172.21.15.122:0/3972770660 wait complete. 2024-03-16T20:46:10.275 INFO:tasks.ceph:Scrubbing osd.3 2024-03-16T20:46:10.275 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-03-16T20:46:10.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.382+0000 7fac219e2640 1 Processor -- start 2024-03-16T20:46:10.387 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.382+0000 7fac219e2640 1 -- start start 2024-03-16T20:46:10.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.382+0000 7fac219e2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac1c144290 0x7fac1c06ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.382+0000 7fac219e2640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fac1c144c00 0x7fac1c0b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.382+0000 7fac219e2640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac1c06daf0 con 0x7fac1c0b90c0 2024-03-16T20:46:10.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.382+0000 7fac219e2640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac1c0b1390 con 0x7fac1c144c00 2024-03-16T20:46:10.388 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.382+0000 7fac219e2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c06dd40 con 0x7fac1c144290 2024-03-16T20:46:10.389 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.382+0000 7fac1affd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac1c144290 0x7fac1c06ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.382+0000 7fac1b7fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fac1c144c00 0x7fac1c0b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.382+0000 7fac1affd640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac1c144290 0x7fac1c06ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:51406/0 (socket says 172.21.15.122:51406) 2024-03-16T20:46:10.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.382+0000 7fac1b7fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fac1c144c00 0x7fac1c0b7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.122:39896/0 (socket says 172.21.15.122:39896) 2024-03-16T20:46:10.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.382+0000 7fac1affd640 1 -- 172.21.15.122:0/3510521662 learned_addr learned my addr 172.21.15.122:0/3510521662 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:10.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.382+0000 7fac1a7fc640 1 --1- >> v1:172.21.15.195:6789/0 conn(0x7fac1c0b90c0 0x7fac1c13df80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.195:6789/0 says I am v1:172.21.15.122:60502/0 (socket says 172.21.15.122:60502) 2024-03-16T20:46:10.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.382+0000 7fac1affd640 1 -- 172.21.15.122:0/3510521662 >> v2:172.21.15.195:3300/0 conn(0x7fac1c144c00 msgr2=0x7fac1c0b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:10.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.382+0000 7fac1affd640 1 --2- 172.21.15.122:0/3510521662 >> v2:172.21.15.195:3300/0 conn(0x7fac1c144c00 0x7fac1c0b7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.390 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.382+0000 7fac1affd640 1 -- 172.21.15.122:0/3510521662 >> v1:172.21.15.195:6789/0 conn(0x7fac1c0b90c0 legacy=0x7fac1c13df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.382+0000 7fac1affd640 1 -- 172.21.15.122:0/3510521662 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c13e690 con 0x7fac1c144290 2024-03-16T20:46:10.391 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.386+0000 7fac1affd640 1 --2- 172.21.15.122:0/3510521662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac1c144290 0x7fac1c06ed70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fac14009e50 tx=0x7fac1401c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b89b8354f13b266 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:10.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.386+0000 7fac19ffb640 1 -- 172.21.15.122:0/3510521662 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac140a55e0 con 0x7fac1c144290 2024-03-16T20:46:10.392 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.386+0000 7fac19ffb640 1 -- 172.21.15.122:0/3510521662 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac140a5740 con 0x7fac1c144290 2024-03-16T20:46:10.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.386+0000 7fac19ffb640 1 -- 172.21.15.122:0/3510521662 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac140a5ad0 con 0x7fac1c144290 2024-03-16T20:46:10.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.386+0000 7fac219e2640 1 -- 172.21.15.122:0/3510521662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac1c144290 msgr2=0x7fac1c06ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.386+0000 7fac219e2640 1 --2- 172.21.15.122:0/3510521662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac1c144290 0x7fac1c06ed70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fac14009e50 tx=0x7fac1401c910 comp rx=0 tx=0).stop 2024-03-16T20:46:10.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.386+0000 7fac219e2640 1 -- 172.21.15.122:0/3510521662 shutdown_connections 2024-03-16T20:46:10.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.386+0000 7fac219e2640 1 --2- 172.21.15.122:0/3510521662 >> v2:172.21.15.195:3300/0 conn(0x7fac1c144c00 0x7fac1c0b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.386+0000 7fac219e2640 1 --2- 172.21.15.122:0/3510521662 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac1c144290 0x7fac1c06ed70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.386+0000 7fac219e2640 1 -- 172.21.15.122:0/3510521662 >> 172.21.15.122:0/3510521662 conn(0x7fac1c070d30 msgr2=0x7fac1c071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:10.393 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.386+0000 7fac219e2640 1 -- 172.21.15.122:0/3510521662 shutdown_connections 2024-03-16T20:46:10.394 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.386+0000 7fac219e2640 1 -- 172.21.15.122:0/3510521662 wait complete. 2024-03-16T20:46:10.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.386+0000 7fac219e2640 1 Processor -- start 2024-03-16T20:46:10.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.386+0000 7fac219e2640 1 -- start start 2024-03-16T20:46:10.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac219e2640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac1c0b90c0 0x7fac1c13d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.398 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac219e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c144290 0x7fac1c145570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac219e2640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac1c144c00 0x7fac1c138690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac219e2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c1471f0 con 0x7fac1c144290 2024-03-16T20:46:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac219e2640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c139b20 con 0x7fac1c144c00 2024-03-16T20:46:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac219e2640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac1c1452c0 con 0x7fac1c0b90c0 2024-03-16T20:46:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac1a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c144290 0x7fac1c145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac1a7fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c144290 0x7fac1c145570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54650/0 (socket says 172.21.15.122:54650) 2024-03-16T20:46:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac1a7fc640 1 -- 172.21.15.122:0/903853690 learned_addr learned my addr 172.21.15.122:0/903853690 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac1b7fe640 1 --2- 172.21.15.122:0/903853690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac1c144c00 0x7fac1c138690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.399 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac1affd640 1 --2- 172.21.15.122:0/903853690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac1c0b90c0 0x7fac1c13d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac1a7fc640 1 -- 172.21.15.122:0/903853690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac1c0b90c0 msgr2=0x7fac1c13d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac1a7fc640 1 --2- 172.21.15.122:0/903853690 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fac1c0b90c0 0x7fac1c13d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac1a7fc640 1 -- 172.21.15.122:0/903853690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac1c144c00 msgr2=0x7fac1c138690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac1a7fc640 1 --2- 172.21.15.122:0/903853690 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fac1c144c00 0x7fac1c138690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac1a7fc640 1 -- 172.21.15.122:0/903853690 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac14009b30 con 0x7fac1c144290 2024-03-16T20:46:10.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac1a7fc640 1 --2- 172.21.15.122:0/903853690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c144290 0x7fac1c145570 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fac0400c370 tx=0x7fac0400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:10.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac13fff640 1 -- 172.21.15.122:0/903853690 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac040074e0 con 0x7fac1c144290 2024-03-16T20:46:10.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac13fff640 1 -- 172.21.15.122:0/903853690 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac04007640 con 0x7fac1c144290 2024-03-16T20:46:10.400 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac219e2640 1 -- 172.21.15.122:0/903853690 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac1c13a780 con 0x7fac1c144290 2024-03-16T20:46:10.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac13fff640 1 -- 172.21.15.122:0/903853690 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac04004a30 con 0x7fac1c144290 2024-03-16T20:46:10.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac219e2640 1 -- 172.21.15.122:0/903853690 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac1c146230 con 0x7fac1c144290 2024-03-16T20:46:10.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac219e2640 1 -- 172.21.15.122:0/903853690 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fac1c145120 con 0x7fac1c144290 2024-03-16T20:46:10.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac13fff640 1 -- 172.21.15.122:0/903853690 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fac04004b90 con 0x7fac1c144290 2024-03-16T20:46:10.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac13fff640 1 --2- 172.21.15.122:0/903853690 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fabfc066e40 0x7fabfc069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac13fff640 1 -- 172.21.15.122:0/903853690 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fac040847a0 con 0x7fac1c144290 2024-03-16T20:46:10.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac13fff640 1 --2- 172.21.15.122:0/903853690 >> [v2:172.21.15.122:6816/362583029,v1:172.21.15.122:6817/362583029] conn(0x7fabfc06cd00 0x7fabfc06f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac13fff640 1 -- 172.21.15.122:0/903853690 --> [v2:172.21.15.122:6816/362583029,v1:172.21.15.122:6817/362583029] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fabfc06f7d0 con 0x7fabfc06cd00 2024-03-16T20:46:10.401 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac13fff640 1 -- 172.21.15.122:0/903853690 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac04084bf0 con 0x7fac1c144290 2024-03-16T20:46:10.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac1affd640 1 --2- 172.21.15.122:0/903853690 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fabfc066e40 0x7fabfc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac1b7fe640 1 --2- 172.21.15.122:0/903853690 >> [v2:172.21.15.122:6816/362583029,v1:172.21.15.122:6817/362583029] conn(0x7fabfc06cd00 0x7fabfc06f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.390+0000 7fac1b7fe640 1 --2- 172.21.15.122:0/903853690 >> [v2:172.21.15.122:6816/362583029,v1:172.21.15.122:6817/362583029] conn(0x7fabfc06cd00 0x7fabfc06f120 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:10.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.394+0000 7fac1affd640 1 --2- 172.21.15.122:0/903853690 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fabfc066e40 0x7fabfc069300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fac140ae000 tx=0x7fac140a7070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:10.402 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.394+0000 7fac13fff640 1 -- 172.21.15.122:0/903853690 <== osd.3 v2:172.21.15.122:6816/362583029 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fabfc06f7d0 con 0x7fabfc06cd00 2024-03-16T20:46:10.411 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.406+0000 7fac219e2640 1 -- 172.21.15.122:0/903853690 --> [v2:172.21.15.122:6816/362583029,v1:172.21.15.122:6817/362583029] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fac1c13aa90 con 0x7fabfc06cd00 2024-03-16T20:46:10.412 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.406+0000 7fac13fff640 1 -- 172.21.15.122:0/903853690 <== osd.3 v2:172.21.15.122:6816/362583029 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fac0c0096a0 con 0x7fabfc06cd00 2024-03-16T20:46:10.412 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-16T20:46:10.412 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T20:46:10.412 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-16T20:46:10.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.406+0000 7fac219e2640 1 -- 172.21.15.122:0/903853690 >> [v2:172.21.15.122:6816/362583029,v1:172.21.15.122:6817/362583029] conn(0x7fabfc06cd00 msgr2=0x7fabfc06f120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.406+0000 7fac219e2640 1 --2- 172.21.15.122:0/903853690 >> [v2:172.21.15.122:6816/362583029,v1:172.21.15.122:6817/362583029] conn(0x7fabfc06cd00 0x7fabfc06f120 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.413 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.406+0000 7fac219e2640 1 -- 172.21.15.122:0/903853690 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fabfc066e40 msgr2=0x7fabfc069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.406+0000 7fac219e2640 1 --2- 172.21.15.122:0/903853690 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fabfc066e40 0x7fabfc069300 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fac140ae000 tx=0x7fac140a7070 comp rx=0 tx=0).stop 2024-03-16T20:46:10.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.406+0000 7fac219e2640 1 -- 172.21.15.122:0/903853690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c144290 msgr2=0x7fac1c145570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.406+0000 7fac219e2640 1 --2- 172.21.15.122:0/903853690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c144290 0x7fac1c145570 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fac0400c370 tx=0x7fac0400c830 comp rx=0 tx=0).stop 2024-03-16T20:46:10.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.406+0000 7fac1b7fe640 1 -- 172.21.15.122:0/903853690 reap_dead start 2024-03-16T20:46:10.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.406+0000 7fac219e2640 1 -- 172.21.15.122:0/903853690 shutdown_connections 2024-03-16T20:46:10.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.406+0000 7fac219e2640 1 -- 172.21.15.122:0/903853690 >> 172.21.15.122:0/903853690 conn(0x7fac1c070d30 msgr2=0x7fac1c0b7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:10.414 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.406+0000 7fac219e2640 1 -- 172.21.15.122:0/903853690 shutdown_connections 2024-03-16T20:46:10.415 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.406+0000 7fac219e2640 1 -- 172.21.15.122:0/903853690 wait complete. 2024-03-16T20:46:10.426 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-03-16T20:46:10.538 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03b576e640 1 Processor -- start 2024-03-16T20:46:10.539 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03b576e640 1 -- start start 2024-03-16T20:46:10.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03b576e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f03b0148f00 0x7f03b013ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03b576e640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f03b013f320 0x7f03b014f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03b576e640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b00b3480 con 0x7f03b00b30a0 2024-03-16T20:46:10.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03b576e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b00b83a0 con 0x7f03b0148f00 2024-03-16T20:46:10.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03b576e640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b00b8680 con 0x7f03b013f320 2024-03-16T20:46:10.540 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03a7fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f03b0148f00 0x7f03b013ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03a7fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f03b0148f00 0x7f03b013ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54652/0 (socket says 172.21.15.122:54652) 2024-03-16T20:46:10.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03a7fff640 1 -- 172.21.15.122:0/745231226 learned_addr learned my addr 172.21.15.122:0/745231226 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:10.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03af7fe640 1 --2- 172.21.15.122:0/745231226 >> v2:172.21.15.195:3300/0 conn(0x7f03b013f320 0x7f03b014f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03a7fff640 1 -- 172.21.15.122:0/745231226 >> v2:172.21.15.195:3300/0 conn(0x7f03b013f320 msgr2=0x7f03b014f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:10.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03a7fff640 1 --2- 172.21.15.122:0/745231226 >> v2:172.21.15.195:3300/0 conn(0x7f03b013f320 0x7f03b014f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03a7fff640 1 -- 172.21.15.122:0/745231226 >> v1:172.21.15.195:6789/0 conn(0x7f03b00b30a0 legacy=0x7f03b01487f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:10.541 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03a7fff640 1 -- 172.21.15.122:0/745231226 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b00b8960 con 0x7f03b0148f00 2024-03-16T20:46:10.542 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03a7fff640 1 --2- 172.21.15.122:0/745231226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b0148f00 0x7f03b013ede0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f03b0073730 tx=0x7f039801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f40139752b8ef3 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:10.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03ae7fc640 1 -- 172.21.15.122:0/745231226 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03980ab600 con 0x7f03b0148f00 2024-03-16T20:46:10.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03ae7fc640 1 -- 172.21.15.122:0/745231226 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03980a7050 con 0x7f03b0148f00 2024-03-16T20:46:10.543 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.534+0000 7f03ae7fc640 1 -- 172.21.15.122:0/745231226 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03980ab980 con 0x7f03b0148f00 2024-03-16T20:46:10.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 -- 172.21.15.122:0/745231226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b0148f00 msgr2=0x7f03b013ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 --2- 172.21.15.122:0/745231226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b0148f00 0x7f03b013ede0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f03b0073730 tx=0x7f039801c450 comp rx=0 tx=0).stop 2024-03-16T20:46:10.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 -- 172.21.15.122:0/745231226 shutdown_connections 2024-03-16T20:46:10.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 --2- 172.21.15.122:0/745231226 >> v2:172.21.15.195:3300/0 conn(0x7f03b013f320 0x7f03b014f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.544 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 --2- 172.21.15.122:0/745231226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b0148f00 0x7f03b013ede0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 -- 172.21.15.122:0/745231226 >> 172.21.15.122:0/745231226 conn(0x7f03b0070d30 msgr2=0x7f03b0071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:10.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 -- 172.21.15.122:0/745231226 shutdown_connections 2024-03-16T20:46:10.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 -- 172.21.15.122:0/745231226 wait complete. 2024-03-16T20:46:10.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 Processor -- start 2024-03-16T20:46:10.545 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 -- start start 2024-03-16T20:46:10.547 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b00b30a0 0x7f03b0135160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03b013f320 0x7f03b00b5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03b0148f00 0x7f03b01dda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b01dd5e0 con 0x7f03b00b30a0 2024-03-16T20:46:10.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b0133ca0 con 0x7f03b013f320 2024-03-16T20:46:10.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b00b48d0 con 0x7f03b0148f00 2024-03-16T20:46:10.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03aeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b00b30a0 0x7f03b0135160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03aeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b00b30a0 0x7f03b0135160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54666/0 (socket says 172.21.15.122:54666) 2024-03-16T20:46:10.548 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03aeffd640 1 -- 172.21.15.122:0/113934987 learned_addr learned my addr 172.21.15.122:0/113934987 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:10.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03a7fff640 1 --2- 172.21.15.122:0/113934987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03b013f320 0x7f03b00b5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03af7fe640 1 --2- 172.21.15.122:0/113934987 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03b0148f00 0x7f03b01dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03aeffd640 1 -- 172.21.15.122:0/113934987 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03b0148f00 msgr2=0x7f03b01dda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03aeffd640 1 --2- 172.21.15.122:0/113934987 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03b0148f00 0x7f03b01dda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03aeffd640 1 -- 172.21.15.122:0/113934987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03b013f320 msgr2=0x7f03b00b5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03aeffd640 1 --2- 172.21.15.122:0/113934987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03b013f320 0x7f03b00b5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03aeffd640 1 -- 172.21.15.122:0/113934987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03980094d0 con 0x7f03b00b30a0 2024-03-16T20:46:10.549 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03a7fff640 1 --2- 172.21.15.122:0/113934987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03b013f320 0x7f03b00b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-16T20:46:10.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03aeffd640 1 --2- 172.21.15.122:0/113934987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b00b30a0 0x7f03b0135160 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f03a000b410 tx=0x7f03a000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:10.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03a77fe640 1 -- 172.21.15.122:0/113934987 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a0004020 con 0x7f03b00b30a0 2024-03-16T20:46:10.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03a77fe640 1 -- 172.21.15.122:0/113934987 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a0004180 con 0x7f03b00b30a0 2024-03-16T20:46:10.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 -- 172.21.15.122:0/113934987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b0148070 con 0x7f03b00b30a0 2024-03-16T20:46:10.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03a77fe640 1 -- 172.21.15.122:0/113934987 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a0011630 con 0x7f03b00b30a0 2024-03-16T20:46:10.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.538+0000 7f03b576e640 1 -- 172.21.15.122:0/113934987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b00b5040 con 0x7f03b00b30a0 2024-03-16T20:46:10.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.542+0000 7f03b576e640 1 -- 172.21.15.122:0/113934987 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b0133b00 con 0x7f03b00b30a0 2024-03-16T20:46:10.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.542+0000 7f03a77fe640 1 -- 172.21.15.122:0/113934987 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f03a0011790 con 0x7f03b00b30a0 2024-03-16T20:46:10.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.542+0000 7f03a77fe640 1 --2- 172.21.15.122:0/113934987 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0390066e70 0x7f0390069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.542+0000 7f03a77fe640 1 -- 172.21.15.122:0/113934987 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f03a0084410 con 0x7f03b00b30a0 2024-03-16T20:46:10.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.542+0000 7f03a7fff640 1 --2- 172.21.15.122:0/113934987 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0390066e70 0x7f0390069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.542+0000 7f03a7fff640 1 --2- 172.21.15.122:0/113934987 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0390066e70 0x7f0390069330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f03980ad000 tx=0x7f03980a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:10.551 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.546+0000 7f03a77fe640 1 -- 172.21.15.122:0/113934987 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f03b0133b00 con 0x7f03b00b30a0 2024-03-16T20:46:10.697 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.690+0000 7f03b576e640 1 -- 172.21.15.122:0/113934987 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f03b01526f0 con 0x7f0390066e70 2024-03-16T20:46:10.698 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.690+0000 7f03a77fe640 1 -- 172.21.15.122:0/113934987 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f03980bb090 con 0x7f0390066e70 2024-03-16T20:46:10.698 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 3 to deep-scrub 2024-03-16T20:46:10.699 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.694+0000 7f03b576e640 1 -- 172.21.15.122:0/113934987 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0390066e70 msgr2=0x7f0390069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.694+0000 7f03b576e640 1 --2- 172.21.15.122:0/113934987 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0390066e70 0x7f0390069330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f03980ad000 tx=0x7f03980a6000 comp rx=0 tx=0).stop 2024-03-16T20:46:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.694+0000 7f03b576e640 1 -- 172.21.15.122:0/113934987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b00b30a0 msgr2=0x7f03b0135160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.694+0000 7f03b576e640 1 --2- 172.21.15.122:0/113934987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b00b30a0 0x7f03b0135160 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f03a000b410 tx=0x7f03a000b8d0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.694+0000 7f03b576e640 1 -- 172.21.15.122:0/113934987 shutdown_connections 2024-03-16T20:46:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.694+0000 7f03b576e640 1 --2- 172.21.15.122:0/113934987 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f0390066e70 0x7f0390069330 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.694+0000 7f03b576e640 1 --2- 172.21.15.122:0/113934987 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f03b0148f00 0x7f03b01dda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.694+0000 7f03b576e640 1 --2- 172.21.15.122:0/113934987 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f03b013f320 0x7f03b00b5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.700 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.694+0000 7f03b576e640 1 --2- 172.21.15.122:0/113934987 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f03b00b30a0 0x7f03b0135160 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.694+0000 7f03b576e640 1 -- 172.21.15.122:0/113934987 >> 172.21.15.122:0/113934987 conn(0x7f03b0070d30 msgr2=0x7f03b006b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:10.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.694+0000 7f03b576e640 1 -- 172.21.15.122:0/113934987 shutdown_connections 2024-03-16T20:46:10.701 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.694+0000 7f03b576e640 1 -- 172.21.15.122:0/113934987 wait complete. 2024-03-16T20:46:10.715 INFO:tasks.ceph:Scrubbing osd.4 2024-03-16T20:46:10.715 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-03-16T20:46:10.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb22b821640 1 Processor -- start 2024-03-16T20:46:10.839 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb22b821640 1 -- start start 2024-03-16T20:46:10.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb22b821640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fb2240b6f00 0x7fb22414f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb22b821640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2240b38e0 con 0x7fb22409ae30 2024-03-16T20:46:10.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb22b821640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb22409b210 con 0x7fb2240b4020 2024-03-16T20:46:10.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb22b821640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2240af790 con 0x7fb2240b6f00 2024-03-16T20:46:10.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb2295bd640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fb22409ae30 0x7fb2240af080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:42162/0 (socket says 172.21.15.122:42162) 2024-03-16T20:46:10.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb2295bd640 1 -- 172.21.15.122:0/2315548436 learned_addr learned my addr 172.21.15.122:0/2315548436 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:46:10.842 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb229dbe640 1 --2- 172.21.15.122:0/2315548436 >> v2:172.21.15.195:3300/0 conn(0x7fb2240b6f00 0x7fb22414f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb21bfff640 1 -- 172.21.15.122:0/2315548436 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702066212 0 0) 0x7fb2240b38e0 con 0x7fb22409ae30 2024-03-16T20:46:10.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb21bfff640 1 -- 172.21.15.122:0/2315548436 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb210003530 con 0x7fb22409ae30 2024-03-16T20:46:10.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb21bfff640 1 -- 172.21.15.122:0/2315548436 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2624254422 0 0) 0x7fb210003530 con 0x7fb22409ae30 2024-03-16T20:46:10.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb21bfff640 1 -- 172.21.15.122:0/2315548436 >> v2:172.21.15.195:3300/0 conn(0x7fb2240b6f00 msgr2=0x7fb22414f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:10.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb21bfff640 1 --2- 172.21.15.122:0/2315548436 >> v2:172.21.15.195:3300/0 conn(0x7fb2240b6f00 0x7fb22414f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb21bfff640 1 -- 172.21.15.122:0/2315548436 >> v1:172.21.15.195:6789/0 conn(0x7fb2240b4020 legacy=0x7fb2240b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb21bfff640 1 -- 172.21.15.122:0/2315548436 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2240b4fb0 con 0x7fb22409ae30 2024-03-16T20:46:10.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb21bfff640 1 -- 172.21.15.122:0/2315548436 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7fb220001680 con 0x7fb22409ae30 2024-03-16T20:46:10.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb21bfff640 1 -- 172.21.15.122:0/2315548436 >> v1:172.21.15.122:6789/0 conn(0x7fb22409ae30 legacy=0x7fb2240af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb21bfff640 1 --2- 172.21.15.122:0/2315548436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2100039d0 0x7fb210003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb21bfff640 1 --2- 172.21.15.122:0/2315548436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2100047d0 0x7fb210004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb21bfff640 1 --2- 172.21.15.122:0/2315548436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2100051e0 0x7fb21000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb21bfff640 1 -- 172.21.15.122:0/2315548436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2240af790 con 0x7fb2100051e0 2024-03-16T20:46:10.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb21bfff640 1 -- 172.21.15.122:0/2315548436 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21000c130 con 0x7fb2100039d0 2024-03-16T20:46:10.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb21bfff640 1 -- 172.21.15.122:0/2315548436 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21000c2f0 con 0x7fb2100047d0 2024-03-16T20:46:10.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb229dbe640 1 --2- 172.21.15.122:0/2315548436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2100051e0 0x7fb21000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb2295bd640 1 --2- 172.21.15.122:0/2315548436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2100039d0 0x7fb210003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb228dbc640 1 --2- 172.21.15.122:0/2315548436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2100047d0 0x7fb210004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb229dbe640 1 -- 172.21.15.122:0/2315548436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2100047d0 msgr2=0x7fb210004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb229dbe640 1 --2- 172.21.15.122:0/2315548436 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2100047d0 0x7fb210004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb229dbe640 1 -- 172.21.15.122:0/2315548436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2100039d0 msgr2=0x7fb210003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb229dbe640 1 --2- 172.21.15.122:0/2315548436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2100039d0 0x7fb210003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb229dbe640 1 -- 172.21.15.122:0/2315548436 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb210003530 con 0x7fb2100051e0 2024-03-16T20:46:10.845 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.834+0000 7fb2295bd640 1 --2- 172.21.15.122:0/2315548436 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2100039d0 0x7fb210003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb229dbe640 1 -- 172.21.15.122:0/2315548436 reap_dead start 2024-03-16T20:46:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb229dbe640 1 --2- 172.21.15.122:0/2315548436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2100051e0 0x7fb21000b990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009f90 tx=0x7fb21c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d861edc428fdb6bf server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb21bfff640 1 -- 172.21.15.122:0/2315548436 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0a5c70 con 0x7fb2100051e0 2024-03-16T20:46:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb21bfff640 1 -- 172.21.15.122:0/2315548436 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21c00f040 con 0x7fb2100051e0 2024-03-16T20:46:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb21bfff640 1 -- 172.21.15.122:0/2315548436 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0039a0 con 0x7fb2100051e0 2024-03-16T20:46:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb22b821640 1 -- 172.21.15.122:0/2315548436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2100051e0 msgr2=0x7fb21000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.850 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb22b821640 1 --2- 172.21.15.122:0/2315548436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2100051e0 0x7fb21000b990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009f90 tx=0x7fb21c01cc60 comp rx=0 tx=0).stop 2024-03-16T20:46:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb22b821640 1 -- 172.21.15.122:0/2315548436 shutdown_connections 2024-03-16T20:46:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb22b821640 1 --2- 172.21.15.122:0/2315548436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2100051e0 0x7fb21000b990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb22b821640 1 -- 172.21.15.122:0/2315548436 >> 172.21.15.122:0/2315548436 conn(0x7fb2240b8d10 msgr2=0x7fb22409a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb22b821640 1 -- 172.21.15.122:0/2315548436 shutdown_connections 2024-03-16T20:46:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb22b821640 1 -- 172.21.15.122:0/2315548436 wait complete. 2024-03-16T20:46:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb22b821640 1 Processor -- start 2024-03-16T20:46:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb22b821640 1 -- start start 2024-03-16T20:46:10.851 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb22b821640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2100051e0 0x7fb2241ce490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb22b821640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2241cf1c0 0x7fb224070b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb22b821640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2241cf640 0x7fb224071090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb22b821640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2241cdab0 con 0x7fb2100051e0 2024-03-16T20:46:10.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb22b821640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2241cd300 con 0x7fb2241cf640 2024-03-16T20:46:10.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb22b821640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2241cfb80 con 0x7fb2241cf1c0 2024-03-16T20:46:10.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb2295bd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2100051e0 0x7fb2241ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb2295bd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2100051e0 0x7fb2241ce490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54690/0 (socket says 172.21.15.122:54690) 2024-03-16T20:46:10.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb2295bd640 1 -- 172.21.15.122:0/1520215368 learned_addr learned my addr 172.21.15.122:0/1520215368 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:10.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb229dbe640 1 --2- 172.21.15.122:0/1520215368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2241cf640 0x7fb224071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb2295bd640 1 -- 172.21.15.122:0/1520215368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2241cf1c0 msgr2=0x7fb224070b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb2295bd640 1 --2- 172.21.15.122:0/1520215368 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fb2241cf1c0 0x7fb224070b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb2295bd640 1 -- 172.21.15.122:0/1520215368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2241cf640 msgr2=0x7fb224071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb2295bd640 1 --2- 172.21.15.122:0/1520215368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2241cf640 0x7fb224071090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb2295bd640 1 -- 172.21.15.122:0/1520215368 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2240b4e20 con 0x7fb2100051e0 2024-03-16T20:46:10.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.838+0000 7fb229dbe640 1 --2- 172.21.15.122:0/1520215368 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fb2241cf640 0x7fb224071090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:10.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.842+0000 7fb2295bd640 1 --2- 172.21.15.122:0/1520215368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2100051e0 0x7fb2241ce490 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb220010b80 tx=0x7fb22000e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:10.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.842+0000 7fb219ffb640 1 -- 172.21.15.122:0/1520215368 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22000ed80 con 0x7fb2100051e0 2024-03-16T20:46:10.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.842+0000 7fb219ffb640 1 -- 172.21.15.122:0/1520215368 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2200143a0 con 0x7fb2100051e0 2024-03-16T20:46:10.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.842+0000 7fb22b821640 1 -- 172.21.15.122:0/1520215368 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2241cfe60 con 0x7fb2100051e0 2024-03-16T20:46:10.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.842+0000 7fb219ffb640 1 -- 172.21.15.122:0/1520215368 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220014710 con 0x7fb2100051e0 2024-03-16T20:46:10.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.842+0000 7fb22b821640 1 -- 172.21.15.122:0/1520215368 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2241dd8d0 con 0x7fb2100051e0 2024-03-16T20:46:10.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.842+0000 7fb22b821640 1 -- 172.21.15.122:0/1520215368 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb2240b3570 con 0x7fb2100051e0 2024-03-16T20:46:10.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.842+0000 7fb219ffb640 1 -- 172.21.15.122:0/1520215368 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fb220014870 con 0x7fb2100051e0 2024-03-16T20:46:10.854 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.842+0000 7fb219ffb640 1 --2- 172.21.15.122:0/1520215368 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb208066e70 0x7fb208069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.842+0000 7fb219ffb640 1 -- 172.21.15.122:0/1520215368 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb220087dc0 con 0x7fb2100051e0 2024-03-16T20:46:10.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.842+0000 7fb219ffb640 1 --2- 172.21.15.122:0/1520215368 >> [v2:172.21.15.195:6807/3059274194,v1:172.21.15.195:6809/3059274194] conn(0x7fb20806cd30 0x7fb20806f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.842+0000 7fb219ffb640 1 -- 172.21.15.122:0/1520215368 --> [v2:172.21.15.195:6807/3059274194,v1:172.21.15.195:6809/3059274194] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb20806f800 con 0x7fb20806cd30 2024-03-16T20:46:10.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.842+0000 7fb219ffb640 1 -- 172.21.15.122:0/1520215368 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb220088210 con 0x7fb2100051e0 2024-03-16T20:46:10.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.842+0000 7fb228dbc640 1 --2- 172.21.15.122:0/1520215368 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb208066e70 0x7fb208069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.842+0000 7fb229dbe640 1 --2- 172.21.15.122:0/1520215368 >> [v2:172.21.15.195:6807/3059274194,v1:172.21.15.195:6809/3059274194] conn(0x7fb20806cd30 0x7fb20806f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.842+0000 7fb228dbc640 1 --2- 172.21.15.122:0/1520215368 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb208066e70 0x7fb208069330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb214007fd0 tx=0x7fb214009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:10.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.842+0000 7fb229dbe640 1 --2- 172.21.15.122:0/1520215368 >> [v2:172.21.15.195:6807/3059274194,v1:172.21.15.195:6809/3059274194] conn(0x7fb20806cd30 0x7fb20806f150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:10.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.846+0000 7fb219ffb640 1 -- 172.21.15.122:0/1520215368 <== osd.4 v2:172.21.15.195:6807/3059274194 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb20806f800 con 0x7fb20806cd30 2024-03-16T20:46:10.863 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.858+0000 7fb22b821640 1 -- 172.21.15.122:0/1520215368 --> [v2:172.21.15.195:6807/3059274194,v1:172.21.15.195:6809/3059274194] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb2241ced70 con 0x7fb20806cd30 2024-03-16T20:46:10.864 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.858+0000 7fb219ffb640 1 -- 172.21.15.122:0/1520215368 <== osd.4 v2:172.21.15.195:6807/3059274194 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb21c0a6cc0 con 0x7fb20806cd30 2024-03-16T20:46:10.865 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-16T20:46:10.865 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T20:46:10.865 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-16T20:46:10.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.858+0000 7fb22b821640 1 -- 172.21.15.122:0/1520215368 >> [v2:172.21.15.195:6807/3059274194,v1:172.21.15.195:6809/3059274194] conn(0x7fb20806cd30 msgr2=0x7fb20806f150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.858+0000 7fb22b821640 1 --2- 172.21.15.122:0/1520215368 >> [v2:172.21.15.195:6807/3059274194,v1:172.21.15.195:6809/3059274194] conn(0x7fb20806cd30 0x7fb20806f150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.858+0000 7fb22b821640 1 -- 172.21.15.122:0/1520215368 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb208066e70 msgr2=0x7fb208069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.866 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.858+0000 7fb22b821640 1 --2- 172.21.15.122:0/1520215368 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fb208066e70 0x7fb208069330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb214007fd0 tx=0x7fb214009040 comp rx=0 tx=0).stop 2024-03-16T20:46:10.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.858+0000 7fb22b821640 1 -- 172.21.15.122:0/1520215368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2100051e0 msgr2=0x7fb2241ce490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.858+0000 7fb22b821640 1 --2- 172.21.15.122:0/1520215368 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2100051e0 0x7fb2241ce490 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fb220010b80 tx=0x7fb22000e4d0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.858+0000 7fb229dbe640 1 -- 172.21.15.122:0/1520215368 reap_dead start 2024-03-16T20:46:10.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.858+0000 7fb22b821640 1 -- 172.21.15.122:0/1520215368 shutdown_connections 2024-03-16T20:46:10.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.858+0000 7fb22b821640 1 -- 172.21.15.122:0/1520215368 >> 172.21.15.122:0/1520215368 conn(0x7fb2240b8d10 msgr2=0x7fb2240b58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:10.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.858+0000 7fb22b821640 1 -- 172.21.15.122:0/1520215368 shutdown_connections 2024-03-16T20:46:10.867 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.858+0000 7fb22b821640 1 -- 172.21.15.122:0/1520215368 wait complete. 2024-03-16T20:46:10.879 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-03-16T20:46:10.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd24dcc0640 1 Processor -- start 2024-03-16T20:46:10.983 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd24dcc0640 1 -- start start 2024-03-16T20:46:10.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd24dcc0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd24809ae30 0x7fd2480af080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd24dcc0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2480b4020 0x7fd2480b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd24dcc0640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2480b38e0 con 0x7fd2480b7820 2024-03-16T20:46:10.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd24dcc0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24809b210 con 0x7fd24809ae30 2024-03-16T20:46:10.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd24dcc0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2480b4fb0 con 0x7fd2480b4020 2024-03-16T20:46:10.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd247fff640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fd2480b7820 0x7fd2481473b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:42176/0 (socket says 172.21.15.122:42176) 2024-03-16T20:46:10.986 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd247fff640 1 -- 172.21.15.122:0/2862842683 learned_addr learned my addr 172.21.15.122:0/2862842683 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:46:10.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2477fe640 1 --2- 172.21.15.122:0/2862842683 >> v2:172.21.15.195:3300/0 conn(0x7fd24809ae30 0x7fd2480af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd246ffd640 1 --2- 172.21.15.122:0/2862842683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2480b4020 0x7fd2480b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2467fc640 1 -- 172.21.15.122:0/2862842683 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1708743590 0 0) 0x7fd2480b38e0 con 0x7fd2480b7820 2024-03-16T20:46:10.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2467fc640 1 -- 172.21.15.122:0/2862842683 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd230003530 con 0x7fd2480b7820 2024-03-16T20:46:10.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2467fc640 1 -- 172.21.15.122:0/2862842683 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3749255562 0 0) 0x7fd230003530 con 0x7fd2480b7820 2024-03-16T20:46:10.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2467fc640 1 -- 172.21.15.122:0/2862842683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2480b4020 msgr2=0x7fd2480b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:10.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2467fc640 1 --2- 172.21.15.122:0/2862842683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2480b4020 0x7fd2480b67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2467fc640 1 -- 172.21.15.122:0/2862842683 >> v2:172.21.15.195:3300/0 conn(0x7fd24809ae30 msgr2=0x7fd2480af080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:10.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2467fc640 1 --2- 172.21.15.122:0/2862842683 >> v2:172.21.15.195:3300/0 conn(0x7fd24809ae30 0x7fd2480af080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2467fc640 1 -- 172.21.15.122:0/2862842683 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2480b3290 con 0x7fd2480b7820 2024-03-16T20:46:10.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2467fc640 1 -- 172.21.15.122:0/2862842683 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7fd240001a00 con 0x7fd2480b7820 2024-03-16T20:46:10.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2467fc640 1 -- 172.21.15.122:0/2862842683 >> v1:172.21.15.122:6789/0 conn(0x7fd2480b7820 legacy=0x7fd2481473b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2467fc640 1 --2- 172.21.15.122:0/2862842683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd230003980 0x7fd230003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2467fc640 1 --2- 172.21.15.122:0/2862842683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2480b4020 0x7fd2300047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2467fc640 1 --2- 172.21.15.122:0/2862842683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd230004d10 0x7fd230005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2467fc640 1 -- 172.21.15.122:0/2862842683 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24809b210 con 0x7fd230004d10 2024-03-16T20:46:10.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2467fc640 1 -- 172.21.15.122:0/2862842683 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2480b4fb0 con 0x7fd230003980 2024-03-16T20:46:10.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2467fc640 1 -- 172.21.15.122:0/2862842683 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23000b990 con 0x7fd2480b4020 2024-03-16T20:46:10.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd247fff640 1 --2- 172.21.15.122:0/2862842683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd230004d10 0x7fd230005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd2477fe640 1 --2- 172.21.15.122:0/2862842683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd230003980 0x7fd230003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd247fff640 1 -- 172.21.15.122:0/2862842683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2480b4020 msgr2=0x7fd2300047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:10.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd247fff640 1 --2- 172.21.15.122:0/2862842683 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2480b4020 0x7fd2300047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd247fff640 1 -- 172.21.15.122:0/2862842683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd230003980 msgr2=0x7fd230003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:10.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd247fff640 1 --2- 172.21.15.122:0/2862842683 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd230003980 0x7fd230003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.978+0000 7fd247fff640 1 -- 172.21.15.122:0/2862842683 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd230003530 con 0x7fd230004d10 2024-03-16T20:46:10.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd247fff640 1 --2- 172.21.15.122:0/2862842683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd230004d10 0x7fd230005190 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd24000a830 tx=0x7fd24001fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffa06e90fd19994 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:10.993 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd2467fc640 1 -- 172.21.15.122:0/2862842683 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2400aa6f0 con 0x7fd230004d10 2024-03-16T20:46:10.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd2467fc640 1 -- 172.21.15.122:0/2862842683 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2400aa850 con 0x7fd230004d10 2024-03-16T20:46:10.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd2467fc640 1 -- 172.21.15.122:0/2862842683 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2400aac00 con 0x7fd230004d10 2024-03-16T20:46:10.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd24dcc0640 1 -- 172.21.15.122:0/2862842683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd230004d10 msgr2=0x7fd230005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd24dcc0640 1 --2- 172.21.15.122:0/2862842683 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd230004d10 0x7fd230005190 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd24000a830 tx=0x7fd24001fac0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd247fff640 1 -- 172.21.15.122:0/2862842683 reap_dead start 2024-03-16T20:46:10.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd24dcc0640 1 -- 172.21.15.122:0/2862842683 shutdown_connections 2024-03-16T20:46:10.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd24dcc0640 1 -- 172.21.15.122:0/2862842683 >> 172.21.15.122:0/2862842683 conn(0x7fd2480b8d10 msgr2=0x7fd24809a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:10.994 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd24dcc0640 1 -- 172.21.15.122:0/2862842683 shutdown_connections 2024-03-16T20:46:10.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd24dcc0640 1 -- 172.21.15.122:0/2862842683 wait complete. 2024-03-16T20:46:10.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd24dcc0640 1 Processor -- start 2024-03-16T20:46:10.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd24dcc0640 1 -- start start 2024-03-16T20:46:10.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd24dcc0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2480995e0 0x7fd248146db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd24dcc0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2481d9150 0x7fd2480706f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd24dcc0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd248140560 0x7fd248140a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd24dcc0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd248070c30 con 0x7fd248140560 2024-03-16T20:46:10.995 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd24dcc0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd248070060 con 0x7fd2481d9150 2024-03-16T20:46:10.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd24dcc0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2481d95b0 con 0x7fd2480995e0 2024-03-16T20:46:10.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd247fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd248140560 0x7fd248140a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd247fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd248140560 0x7fd248140a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54708/0 (socket says 172.21.15.122:54708) 2024-03-16T20:46:10.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd247fff640 1 -- 172.21.15.122:0/3420235187 learned_addr learned my addr 172.21.15.122:0/3420235187 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:10.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd246ffd640 1 --2- 172.21.15.122:0/3420235187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2481d9150 0x7fd2480706f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd2477fe640 1 --2- 172.21.15.122:0/3420235187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2480995e0 0x7fd248146db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd247fff640 1 -- 172.21.15.122:0/3420235187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2480995e0 msgr2=0x7fd248146db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd247fff640 1 --2- 172.21.15.122:0/3420235187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2480995e0 0x7fd248146db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.996 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd247fff640 1 -- 172.21.15.122:0/3420235187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2481d9150 msgr2=0x7fd2480706f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:10.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd247fff640 1 --2- 172.21.15.122:0/3420235187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2481d9150 0x7fd2480706f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:10.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd247fff640 1 -- 172.21.15.122:0/3420235187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2480999c0 con 0x7fd248140560 2024-03-16T20:46:10.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd247fff640 1 --2- 172.21.15.122:0/3420235187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd248140560 0x7fd248140a10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd24009c200 tx=0x7fd230005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:10.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.982+0000 7fd22ffff640 1 -- 172.21.15.122:0/3420235187 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2400aa6f0 con 0x7fd248140560 2024-03-16T20:46:10.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.986+0000 7fd22ffff640 1 -- 172.21.15.122:0/3420235187 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd240002000 con 0x7fd248140560 2024-03-16T20:46:10.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.986+0000 7fd24dcc0640 1 -- 172.21.15.122:0/3420235187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2481d8950 con 0x7fd248140560 2024-03-16T20:46:10.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.986+0000 7fd22ffff640 1 -- 172.21.15.122:0/3420235187 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2400019c0 con 0x7fd248140560 2024-03-16T20:46:10.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.986+0000 7fd24dcc0640 1 -- 172.21.15.122:0/3420235187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd248070260 con 0x7fd248140560 2024-03-16T20:46:10.997 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.986+0000 7fd24dcc0640 1 -- 172.21.15.122:0/3420235187 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2481d8e70 con 0x7fd248140560 2024-03-16T20:46:10.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.986+0000 7fd22ffff640 1 -- 172.21.15.122:0/3420235187 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fd2400c3460 con 0x7fd248140560 2024-03-16T20:46:10.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.986+0000 7fd22ffff640 1 --2- 172.21.15.122:0/3420235187 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd234066e70 0x7fd234069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:10.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.986+0000 7fd22ffff640 1 -- 172.21.15.122:0/3420235187 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd2401249e0 con 0x7fd248140560 2024-03-16T20:46:10.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.986+0000 7fd2477fe640 1 --2- 172.21.15.122:0/3420235187 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd234066e70 0x7fd234069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:10.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.986+0000 7fd2477fe640 1 --2- 172.21.15.122:0/3420235187 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd234066e70 0x7fd234069330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd238009a10 tx=0x7fd238008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:10.998 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:10.990+0000 7fd22ffff640 1 -- 172.21.15.122:0/3420235187 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fd2481d8e70 con 0x7fd248140560 2024-03-16T20:46:11.153 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.146+0000 7fd24dcc0640 1 -- 172.21.15.122:0/3420235187 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd248147ac0 con 0x7fd234066e70 2024-03-16T20:46:11.154 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.146+0000 7fd22ffff640 1 -- 172.21.15.122:0/3420235187 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd238004020 con 0x7fd234066e70 2024-03-16T20:46:11.154 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 4 to deep-scrub 2024-03-16T20:46:11.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.150+0000 7fd24dcc0640 1 -- 172.21.15.122:0/3420235187 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd234066e70 msgr2=0x7fd234069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.150+0000 7fd24dcc0640 1 --2- 172.21.15.122:0/3420235187 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd234066e70 0x7fd234069330 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd238009a10 tx=0x7fd238008040 comp rx=0 tx=0).stop 2024-03-16T20:46:11.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.150+0000 7fd24dcc0640 1 -- 172.21.15.122:0/3420235187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd248140560 msgr2=0x7fd248140a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.156 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.150+0000 7fd24dcc0640 1 --2- 172.21.15.122:0/3420235187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd248140560 0x7fd248140a10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fd24009c200 tx=0x7fd230005800 comp rx=0 tx=0).stop 2024-03-16T20:46:11.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.150+0000 7fd24dcc0640 1 -- 172.21.15.122:0/3420235187 shutdown_connections 2024-03-16T20:46:11.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.150+0000 7fd24dcc0640 1 --2- 172.21.15.122:0/3420235187 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd234066e70 0x7fd234069330 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.150+0000 7fd24dcc0640 1 --2- 172.21.15.122:0/3420235187 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd248140560 0x7fd248140a10 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.150+0000 7fd24dcc0640 1 --2- 172.21.15.122:0/3420235187 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd2481d9150 0x7fd2480706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.150+0000 7fd24dcc0640 1 --2- 172.21.15.122:0/3420235187 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd2480995e0 0x7fd248146db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.150+0000 7fd24dcc0640 1 -- 172.21.15.122:0/3420235187 >> 172.21.15.122:0/3420235187 conn(0x7fd2480b8d10 msgr2=0x7fd2480b6eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:11.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.150+0000 7fd24dcc0640 1 -- 172.21.15.122:0/3420235187 shutdown_connections 2024-03-16T20:46:11.157 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.150+0000 7fd24dcc0640 1 -- 172.21.15.122:0/3420235187 wait complete. 2024-03-16T20:46:11.171 INFO:tasks.ceph:Scrubbing osd.5 2024-03-16T20:46:11.172 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-03-16T20:46:11.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.290+0000 7fc7a922e640 1 Processor -- start 2024-03-16T20:46:11.295 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.290+0000 7fc7a922e640 1 -- start start 2024-03-16T20:46:11.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.290+0000 7fc7a922e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7a406b1f0 0x7fc7a40b7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.290+0000 7fc7a922e640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc7a406f6f0 0x7fc7a4141270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.290+0000 7fc7a922e640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7a4147ca0 con 0x7fc7a41417b0 2024-03-16T20:46:11.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.290+0000 7fc7a922e640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a406b5d0 con 0x7fc7a406f6f0 2024-03-16T20:46:11.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.290+0000 7fc7a922e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a4147650 con 0x7fc7a406b1f0 2024-03-16T20:46:11.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.290+0000 7fc7a259c640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fc7a406f6f0 0x7fc7a4141270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.290+0000 7fc7a359e640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7fc7a41417b0 0x7fc7a414a870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:42180/0 (socket says 172.21.15.122:42180) 2024-03-16T20:46:11.297 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.290+0000 7fc7a359e640 1 -- 172.21.15.122:0/4187270946 learned_addr learned my addr 172.21.15.122:0/4187270946 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:46:11.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.290+0000 7fc7a2d9d640 1 --2- 172.21.15.122:0/4187270946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7a406b1f0 0x7fc7a40b7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.290+0000 7fc7a259c640 1 -- 172.21.15.122:0/4187270946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7a406b1f0 msgr2=0x7fc7a40b7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:11.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.290+0000 7fc7a259c640 1 --2- 172.21.15.122:0/4187270946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7a406b1f0 0x7fc7a40b7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.290+0000 7fc7a259c640 1 -- 172.21.15.122:0/4187270946 >> v1:172.21.15.122:6789/0 conn(0x7fc7a41417b0 legacy=0x7fc7a414a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.290+0000 7fc7a1d9b640 1 -- 172.21.15.122:0/4187270946 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 313490323 0 0) 0x7fc7a4147ca0 con 0x7fc7a41417b0 2024-03-16T20:46:11.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.290+0000 7fc7a259c640 1 -- 172.21.15.122:0/4187270946 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a4147930 con 0x7fc7a406f6f0 2024-03-16T20:46:11.298 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.290+0000 7fc7a2d9d640 1 --2- 172.21.15.122:0/4187270946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7a406b1f0 0x7fc7a40b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:11.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a259c640 1 --2- 172.21.15.122:0/4187270946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7a406f6f0 0x7fc7a4141270 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc7980097c0 tx=0x7fc79801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f890bbe2e9ff49b8 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:11.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a1d9b640 1 -- 172.21.15.122:0/4187270946 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7980ab600 con 0x7fc7a406f6f0 2024-03-16T20:46:11.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a1d9b640 1 -- 172.21.15.122:0/4187270946 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7980a7050 con 0x7fc7a406f6f0 2024-03-16T20:46:11.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a1d9b640 1 -- 172.21.15.122:0/4187270946 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7980ab980 con 0x7fc7a406f6f0 2024-03-16T20:46:11.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a922e640 1 -- 172.21.15.122:0/4187270946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7a406f6f0 msgr2=0x7fc7a4141270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a922e640 1 --2- 172.21.15.122:0/4187270946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7a406f6f0 0x7fc7a4141270 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc7980097c0 tx=0x7fc79801c440 comp rx=0 tx=0).stop 2024-03-16T20:46:11.300 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a922e640 1 -- 172.21.15.122:0/4187270946 shutdown_connections 2024-03-16T20:46:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a922e640 1 --2- 172.21.15.122:0/4187270946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7a406f6f0 0x7fc7a4141270 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a922e640 1 --2- 172.21.15.122:0/4187270946 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7a406b1f0 0x7fc7a40b7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a922e640 1 -- 172.21.15.122:0/4187270946 >> 172.21.15.122:0/4187270946 conn(0x7fc7a4070d30 msgr2=0x7fc7a4071140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a922e640 1 -- 172.21.15.122:0/4187270946 shutdown_connections 2024-03-16T20:46:11.301 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a922e640 1 -- 172.21.15.122:0/4187270946 wait complete. 2024-03-16T20:46:11.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a922e640 1 Processor -- start 2024-03-16T20:46:11.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a922e640 1 -- start start 2024-03-16T20:46:11.304 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a922e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7a406b1f0 0x7fc7a41486c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a922e640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7a406f6f0 0x7fc7a41dfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a922e640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7a41417b0 0x7fc7a41e0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a922e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a413ccc0 con 0x7fc7a406b1f0 2024-03-16T20:46:11.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a922e640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a41e0750 con 0x7fc7a41417b0 2024-03-16T20:46:11.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a922e640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a413c790 con 0x7fc7a406f6f0 2024-03-16T20:46:11.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a2d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7a406b1f0 0x7fc7a41486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a2d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7a406b1f0 0x7fc7a41486c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54732/0 (socket says 172.21.15.122:54732) 2024-03-16T20:46:11.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a2d9d640 1 -- 172.21.15.122:0/387720492 learned_addr learned my addr 172.21.15.122:0/387720492 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:11.305 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a359e640 1 --2- 172.21.15.122:0/387720492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7a41417b0 0x7fc7a41e0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a2d9d640 1 -- 172.21.15.122:0/387720492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7a406f6f0 msgr2=0x7fc7a41dfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a259c640 1 --2- 172.21.15.122:0/387720492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7a406f6f0 0x7fc7a41dfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a2d9d640 1 --2- 172.21.15.122:0/387720492 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fc7a406f6f0 0x7fc7a41dfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a2d9d640 1 -- 172.21.15.122:0/387720492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7a41417b0 msgr2=0x7fc7a41e0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a2d9d640 1 --2- 172.21.15.122:0/387720492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7a41417b0 0x7fc7a41e0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a2d9d640 1 -- 172.21.15.122:0/387720492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc798009420 con 0x7fc7a406b1f0 2024-03-16T20:46:11.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a359e640 1 --2- 172.21.15.122:0/387720492 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fc7a41417b0 0x7fc7a41e0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:11.306 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a2d9d640 1 --2- 172.21.15.122:0/387720492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7a406b1f0 0x7fc7a41486c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00cf60 tx=0x7fc79c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:11.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc78b7fe640 1 -- 172.21.15.122:0/387720492 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c00d840 con 0x7fc7a406b1f0 2024-03-16T20:46:11.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc78b7fe640 1 -- 172.21.15.122:0/387720492 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79c00d9a0 con 0x7fc7a406b1f0 2024-03-16T20:46:11.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.294+0000 7fc7a922e640 1 -- 172.21.15.122:0/387720492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a4148f10 con 0x7fc7a406b1f0 2024-03-16T20:46:11.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.298+0000 7fc78b7fe640 1 -- 172.21.15.122:0/387720492 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc79c00dd30 con 0x7fc7a406b1f0 2024-03-16T20:46:11.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.298+0000 7fc7a922e640 1 -- 172.21.15.122:0/387720492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a413d730 con 0x7fc7a406b1f0 2024-03-16T20:46:11.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.298+0000 7fc7a922e640 1 -- 172.21.15.122:0/387720492 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc7a4140490 con 0x7fc7a406b1f0 2024-03-16T20:46:11.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.298+0000 7fc78b7fe640 1 -- 172.21.15.122:0/387720492 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fc79c015440 con 0x7fc7a406b1f0 2024-03-16T20:46:11.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.298+0000 7fc78b7fe640 1 --2- 172.21.15.122:0/387720492 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc790066e40 0x7fc790069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.298+0000 7fc78b7fe640 1 -- 172.21.15.122:0/387720492 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc79c085c00 con 0x7fc7a406b1f0 2024-03-16T20:46:11.307 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.298+0000 7fc78b7fe640 1 --2- 172.21.15.122:0/387720492 >> [v2:172.21.15.195:6800/1568352129,v1:172.21.15.195:6801/1568352129] conn(0x7fc79006cd00 0x7fc79006f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.298+0000 7fc78b7fe640 1 -- 172.21.15.122:0/387720492 --> [v2:172.21.15.195:6800/1568352129,v1:172.21.15.195:6801/1568352129] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc79006f7d0 con 0x7fc79006cd00 2024-03-16T20:46:11.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.298+0000 7fc78b7fe640 1 -- 172.21.15.122:0/387720492 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc79c086050 con 0x7fc7a406b1f0 2024-03-16T20:46:11.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.298+0000 7fc7a259c640 1 --2- 172.21.15.122:0/387720492 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc790066e40 0x7fc790069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.298+0000 7fc7a359e640 1 --2- 172.21.15.122:0/387720492 >> [v2:172.21.15.195:6800/1568352129,v1:172.21.15.195:6801/1568352129] conn(0x7fc79006cd00 0x7fc79006f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.308 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.298+0000 7fc7a259c640 1 --2- 172.21.15.122:0/387720492 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc790066e40 0x7fc790069300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc7980ad000 tx=0x7fc7980a6000 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:11.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.298+0000 7fc7a359e640 1 --2- 172.21.15.122:0/387720492 >> [v2:172.21.15.195:6800/1568352129,v1:172.21.15.195:6801/1568352129] conn(0x7fc79006cd00 0x7fc79006f120 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:11.309 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.302+0000 7fc78b7fe640 1 -- 172.21.15.122:0/387720492 <== osd.5 v2:172.21.15.195:6800/1568352129 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc79006f7d0 con 0x7fc79006cd00 2024-03-16T20:46:11.320 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.314+0000 7fc7a922e640 1 -- 172.21.15.122:0/387720492 --> [v2:172.21.15.195:6800/1568352129,v1:172.21.15.195:6801/1568352129] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc7a406be80 con 0x7fc79006cd00 2024-03-16T20:46:11.321 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.314+0000 7fc78b7fe640 1 -- 172.21.15.122:0/387720492 <== osd.5 v2:172.21.15.195:6800/1568352129 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc7a406be80 con 0x7fc79006cd00 2024-03-16T20:46:11.321 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-16T20:46:11.321 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T20:46:11.321 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-16T20:46:11.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.314+0000 7fc7a922e640 1 -- 172.21.15.122:0/387720492 >> [v2:172.21.15.195:6800/1568352129,v1:172.21.15.195:6801/1568352129] conn(0x7fc79006cd00 msgr2=0x7fc79006f120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.322 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.314+0000 7fc7a922e640 1 --2- 172.21.15.122:0/387720492 >> [v2:172.21.15.195:6800/1568352129,v1:172.21.15.195:6801/1568352129] conn(0x7fc79006cd00 0x7fc79006f120 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.314+0000 7fc7a922e640 1 -- 172.21.15.122:0/387720492 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc790066e40 msgr2=0x7fc790069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.314+0000 7fc7a922e640 1 --2- 172.21.15.122:0/387720492 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fc790066e40 0x7fc790069300 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc7980ad000 tx=0x7fc7980a6000 comp rx=0 tx=0).stop 2024-03-16T20:46:11.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.314+0000 7fc7a922e640 1 -- 172.21.15.122:0/387720492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7a406b1f0 msgr2=0x7fc7a41486c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.314+0000 7fc7a922e640 1 --2- 172.21.15.122:0/387720492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7a406b1f0 0x7fc7a41486c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc79c00cf60 tx=0x7fc79c00cf90 comp rx=0 tx=0).stop 2024-03-16T20:46:11.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.314+0000 7fc7a359e640 1 -- 172.21.15.122:0/387720492 reap_dead start 2024-03-16T20:46:11.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.314+0000 7fc7a922e640 1 -- 172.21.15.122:0/387720492 shutdown_connections 2024-03-16T20:46:11.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.314+0000 7fc7a922e640 1 -- 172.21.15.122:0/387720492 >> 172.21.15.122:0/387720492 conn(0x7fc7a4070d30 msgr2=0x7fc7a40b8650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:11.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.314+0000 7fc7a922e640 1 -- 172.21.15.122:0/387720492 shutdown_connections 2024-03-16T20:46:11.323 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.314+0000 7fc7a922e640 1 -- 172.21.15.122:0/387720492 wait complete. 2024-03-16T20:46:11.336 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-03-16T20:46:11.446 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a108fe640 1 Processor -- start 2024-03-16T20:46:11.447 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a108fe640 1 -- start start 2024-03-16T20:46:11.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a108fe640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f3a0c0bb1d0 0x7f3a0c06cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a108fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0c06d180 0x7f3a0c151920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a108fe640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a0c147ff0 con 0x7f3a0c141580 2024-03-16T20:46:11.448 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a108fe640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a0c1479a0 con 0x7f3a0c0bb1d0 2024-03-16T20:46:11.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a108fe640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c147c80 con 0x7f3a0c06d180 2024-03-16T20:46:11.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a03fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f3a0c0bb1d0 0x7f3a0c06cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a03fff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f3a0c0bb1d0 0x7f3a0c06cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54744/0 (socket says 172.21.15.122:54744) 2024-03-16T20:46:11.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a03fff640 1 -- 172.21.15.122:0/2994064230 learned_addr learned my addr 172.21.15.122:0/2994064230 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:11.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a0ad9d640 1 --2- 172.21.15.122:0/2994064230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0c06d180 0x7f3a0c151920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a03fff640 1 -- 172.21.15.122:0/2994064230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0c06d180 msgr2=0x7f3a0c151920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:11.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a03fff640 1 --2- 172.21.15.122:0/2994064230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0c06d180 0x7f3a0c151920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.449 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a03fff640 1 -- 172.21.15.122:0/2994064230 >> v1:172.21.15.195:6789/0 conn(0x7f3a0c141580 legacy=0x7f3a0c0baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:11.450 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a03fff640 1 -- 172.21.15.122:0/2994064230 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a0c1475b0 con 0x7f3a0c0bb1d0 2024-03-16T20:46:11.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a03fff640 1 --2- 172.21.15.122:0/2994064230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a0c0bb1d0 0x7f3a0c06cc40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009870 tx=0x7f39f401c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce222abd11e971 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:11.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a09d9b640 1 -- 172.21.15.122:0/2994064230 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f40ab600 con 0x7f3a0c0bb1d0 2024-03-16T20:46:11.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a09d9b640 1 -- 172.21.15.122:0/2994064230 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39f40a7050 con 0x7f3a0c0bb1d0 2024-03-16T20:46:11.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.442+0000 7f3a09d9b640 1 -- 172.21.15.122:0/2994064230 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f40ab980 con 0x7f3a0c0bb1d0 2024-03-16T20:46:11.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 -- 172.21.15.122:0/2994064230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a0c0bb1d0 msgr2=0x7f3a0c06cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 --2- 172.21.15.122:0/2994064230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a0c0bb1d0 0x7f3a0c06cc40 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f39f4009870 tx=0x7f39f401c520 comp rx=0 tx=0).stop 2024-03-16T20:46:11.451 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 -- 172.21.15.122:0/2994064230 shutdown_connections 2024-03-16T20:46:11.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 --2- 172.21.15.122:0/2994064230 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0c06d180 0x7f3a0c151920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 --2- 172.21.15.122:0/2994064230 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a0c0bb1d0 0x7f3a0c06cc40 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 -- 172.21.15.122:0/2994064230 >> 172.21.15.122:0/2994064230 conn(0x7f3a0c0bca20 msgr2=0x7f3a0c0bce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:11.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 -- 172.21.15.122:0/2994064230 shutdown_connections 2024-03-16T20:46:11.452 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 -- 172.21.15.122:0/2994064230 wait complete. 2024-03-16T20:46:11.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 Processor -- start 2024-03-16T20:46:11.454 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 -- start start 2024-03-16T20:46:11.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a0c06d180 0x7f3a0c144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a0c0bb1d0 0x7f3a0c1e34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0c141580 0x7f3a0c13aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c146e20 con 0x7f3a0c06d180 2024-03-16T20:46:11.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c1e3a20 con 0x7f3a0c0bb1d0 2024-03-16T20:46:11.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c0b5a40 con 0x7f3a0c141580 2024-03-16T20:46:11.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a0a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a0c06d180 0x7f3a0c144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.455 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a0a59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a0c06d180 0x7f3a0c144ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54750/0 (socket says 172.21.15.122:54750) 2024-03-16T20:46:11.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a0a59c640 1 -- 172.21.15.122:0/1964857570 learned_addr learned my addr 172.21.15.122:0/1964857570 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:11.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a0ad9d640 1 --2- 172.21.15.122:0/1964857570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0c141580 0x7f3a0c13aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a03fff640 1 --2- 172.21.15.122:0/1964857570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a0c0bb1d0 0x7f3a0c1e34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a0a59c640 1 -- 172.21.15.122:0/1964857570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0c141580 msgr2=0x7f3a0c13aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a0a59c640 1 --2- 172.21.15.122:0/1964857570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0c141580 0x7f3a0c13aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a0a59c640 1 -- 172.21.15.122:0/1964857570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a0c0bb1d0 msgr2=0x7f3a0c1e34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a0a59c640 1 --2- 172.21.15.122:0/1964857570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a0c0bb1d0 0x7f3a0c1e34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.456 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a0a59c640 1 -- 172.21.15.122:0/1964857570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f40094d0 con 0x7f3a0c06d180 2024-03-16T20:46:11.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a03fff640 1 --2- 172.21.15.122:0/1964857570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a0c0bb1d0 0x7f3a0c1e34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:11.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a0a59c640 1 --2- 172.21.15.122:0/1964857570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a0c06d180 0x7f3a0c144ed0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00b440 tx=0x7f39fc00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:11.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a02ffd640 1 -- 172.21.15.122:0/1964857570 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc011890 con 0x7f3a0c06d180 2024-03-16T20:46:11.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a02ffd640 1 -- 172.21.15.122:0/1964857570 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39fc0119f0 con 0x7f3a0c06d180 2024-03-16T20:46:11.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 -- 172.21.15.122:0/1964857570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a0c0b5c90 con 0x7f3a0c06d180 2024-03-16T20:46:11.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a02ffd640 1 -- 172.21.15.122:0/1964857570 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39fc011da0 con 0x7f3a0c06d180 2024-03-16T20:46:11.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.446+0000 7f3a108fe640 1 -- 172.21.15.122:0/1964857570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0c0b53a0 con 0x7f3a0c06d180 2024-03-16T20:46:11.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.450+0000 7f3a108fe640 1 -- 172.21.15.122:0/1964857570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a0c1e3b70 con 0x7f3a0c06d180 2024-03-16T20:46:11.457 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.450+0000 7f3a02ffd640 1 -- 172.21.15.122:0/1964857570 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f39fc01a400 con 0x7f3a0c06d180 2024-03-16T20:46:11.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.450+0000 7f3a02ffd640 1 --2- 172.21.15.122:0/1964857570 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f39ec066e40 0x7f39ec069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.450+0000 7f3a02ffd640 1 -- 172.21.15.122:0/1964857570 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f39fc084710 con 0x7f3a0c06d180 2024-03-16T20:46:11.458 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.450+0000 7f3a03fff640 1 --2- 172.21.15.122:0/1964857570 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f39ec066e40 0x7f39ec069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.450+0000 7f3a03fff640 1 --2- 172.21.15.122:0/1964857570 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f39ec066e40 0x7f39ec069300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f39f40ad000 tx=0x7f39f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:11.459 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.454+0000 7f3a02ffd640 1 -- 172.21.15.122:0/1964857570 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f39fc051890 con 0x7f3a0c06d180 2024-03-16T20:46:11.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.602+0000 7f3a108fe640 1 -- 172.21.15.122:0/1964857570 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a0c0a1eb0 con 0x7f39ec066e40 2024-03-16T20:46:11.608 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.602+0000 7f3a02ffd640 1 -- 172.21.15.122:0/1964857570 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f39f40ba090 con 0x7f39ec066e40 2024-03-16T20:46:11.608 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 5 to deep-scrub 2024-03-16T20:46:11.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.606+0000 7f3a108fe640 1 -- 172.21.15.122:0/1964857570 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f39ec066e40 msgr2=0x7f39ec069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.606+0000 7f3a108fe640 1 --2- 172.21.15.122:0/1964857570 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f39ec066e40 0x7f39ec069300 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f39f40ad000 tx=0x7f39f4008040 comp rx=0 tx=0).stop 2024-03-16T20:46:11.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.606+0000 7f3a108fe640 1 -- 172.21.15.122:0/1964857570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a0c06d180 msgr2=0x7f3a0c144ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.606+0000 7f3a108fe640 1 --2- 172.21.15.122:0/1964857570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a0c06d180 0x7f3a0c144ed0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f39fc00b440 tx=0x7f39fc00b900 comp rx=0 tx=0).stop 2024-03-16T20:46:11.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.606+0000 7f3a108fe640 1 -- 172.21.15.122:0/1964857570 shutdown_connections 2024-03-16T20:46:11.611 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.606+0000 7f3a108fe640 1 --2- 172.21.15.122:0/1964857570 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f39ec066e40 0x7f39ec069300 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.606+0000 7f3a108fe640 1 --2- 172.21.15.122:0/1964857570 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f3a0c141580 0x7f3a0c13aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.606+0000 7f3a108fe640 1 --2- 172.21.15.122:0/1964857570 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f3a0c0bb1d0 0x7f3a0c1e34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.606+0000 7f3a108fe640 1 --2- 172.21.15.122:0/1964857570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3a0c06d180 0x7f3a0c144ed0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.606+0000 7f3a108fe640 1 -- 172.21.15.122:0/1964857570 >> 172.21.15.122:0/1964857570 conn(0x7f3a0c0bca20 msgr2=0x7f3a0c13ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:11.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.606+0000 7f3a108fe640 1 -- 172.21.15.122:0/1964857570 shutdown_connections 2024-03-16T20:46:11.612 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.606+0000 7f3a108fe640 1 -- 172.21.15.122:0/1964857570 wait complete. 2024-03-16T20:46:11.626 INFO:tasks.ceph:Scrubbing osd.6 2024-03-16T20:46:11.626 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-03-16T20:46:11.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd41eed0640 1 Processor -- start 2024-03-16T20:46:11.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd41eed0640 1 -- start start 2024-03-16T20:46:11.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd41eed0640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fd41809ae30 0x7fd4180b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd41eed0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4180aef80 0x7fd418144390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd41eed0640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd4180af3e0 0x7fd4181448d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd41eed0640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4180b3110 con 0x7fd4180af3e0 2024-03-16T20:46:11.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd41eed0640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4180b3970 con 0x7fd41809ae30 2024-03-16T20:46:11.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd41eed0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd41809b210 con 0x7fd4180aef80 2024-03-16T20:46:11.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd41d46d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd4180af3e0 0x7fd4181448d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.741 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd41d46d640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fd4180af3e0 0x7fd4181448d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54758/0 (socket says 172.21.15.122:54758) 2024-03-16T20:46:11.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd41d46d640 1 -- 172.21.15.122:0/319333872 learned_addr learned my addr 172.21.15.122:0/319333872 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:11.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd41cc6c640 1 --2- 172.21.15.122:0/319333872 >> v2:172.21.15.195:3300/0 conn(0x7fd41809ae30 0x7fd4180b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd417fff640 1 --2- 172.21.15.122:0/319333872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4180aef80 0x7fd418144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd41d46d640 1 -- 172.21.15.122:0/319333872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4180aef80 msgr2=0x7fd418144390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:11.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd41d46d640 1 --2- 172.21.15.122:0/319333872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4180aef80 0x7fd418144390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd41d46d640 1 -- 172.21.15.122:0/319333872 >> v2:172.21.15.195:3300/0 conn(0x7fd41809ae30 msgr2=0x7fd4180b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:11.742 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd41d46d640 1 --2- 172.21.15.122:0/319333872 >> v2:172.21.15.195:3300/0 conn(0x7fd41809ae30 0x7fd4180b3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd41d46d640 1 -- 172.21.15.122:0/319333872 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4180b3320 con 0x7fd4180af3e0 2024-03-16T20:46:11.743 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.734+0000 7fd417fff640 1 --2- 172.21.15.122:0/319333872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4180aef80 0x7fd418144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:11.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41d46d640 1 --2- 172.21.15.122:0/319333872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4180af3e0 0x7fd4181448d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd4180b95b0 tx=0x7fd40801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c2b15a0b0880f46 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:11.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd4177fe640 1 -- 172.21.15.122:0/319333872 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4080a56f0 con 0x7fd4180af3e0 2024-03-16T20:46:11.744 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd4177fe640 1 -- 172.21.15.122:0/319333872 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4080a5850 con 0x7fd4180af3e0 2024-03-16T20:46:11.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd4177fe640 1 -- 172.21.15.122:0/319333872 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4080a5be0 con 0x7fd4180af3e0 2024-03-16T20:46:11.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 -- 172.21.15.122:0/319333872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4180af3e0 msgr2=0x7fd4181448d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 --2- 172.21.15.122:0/319333872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4180af3e0 0x7fd4181448d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd4180b95b0 tx=0x7fd40801c960 comp rx=0 tx=0).stop 2024-03-16T20:46:11.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 -- 172.21.15.122:0/319333872 shutdown_connections 2024-03-16T20:46:11.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 --2- 172.21.15.122:0/319333872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4180af3e0 0x7fd4181448d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 --2- 172.21.15.122:0/319333872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4180aef80 0x7fd418144390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 --2- 172.21.15.122:0/319333872 >> v2:172.21.15.195:3300/0 conn(0x7fd41809ae30 0x7fd4180b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 -- 172.21.15.122:0/319333872 >> 172.21.15.122:0/319333872 conn(0x7fd4180b8d10 msgr2=0x7fd41809a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:11.745 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 -- 172.21.15.122:0/319333872 shutdown_connections 2024-03-16T20:46:11.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 -- 172.21.15.122:0/319333872 wait complete. 2024-03-16T20:46:11.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 Processor -- start 2024-03-16T20:46:11.746 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 -- start start 2024-03-16T20:46:11.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd41809ae30 0x7fd4180b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4180aef80 0x7fd4180701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4180af3e0 0x7fd4181467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41806fb00 con 0x7fd4180af3e0 2024-03-16T20:46:11.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd418070710 con 0x7fd41809ae30 2024-03-16T20:46:11.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4180709e0 con 0x7fd4180aef80 2024-03-16T20:46:11.749 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41d46d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4180af3e0 0x7fd4181467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41d46d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4180af3e0 0x7fd4181467c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54766/0 (socket says 172.21.15.122:54766) 2024-03-16T20:46:11.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41d46d640 1 -- 172.21.15.122:0/3665143777 learned_addr learned my addr 172.21.15.122:0/3665143777 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:11.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41cc6c640 1 --2- 172.21.15.122:0/3665143777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd41809ae30 0x7fd4180b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd417fff640 1 --2- 172.21.15.122:0/3665143777 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4180aef80 0x7fd4180701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41d46d640 1 -- 172.21.15.122:0/3665143777 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4180aef80 msgr2=0x7fd4180701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41d46d640 1 --2- 172.21.15.122:0/3665143777 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fd4180aef80 0x7fd4180701d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41d46d640 1 -- 172.21.15.122:0/3665143777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd41809ae30 msgr2=0x7fd4180b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41d46d640 1 --2- 172.21.15.122:0/3665143777 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fd41809ae30 0x7fd4180b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41d46d640 1 -- 172.21.15.122:0/3665143777 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd408009b30 con 0x7fd4180af3e0 2024-03-16T20:46:11.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41d46d640 1 --2- 172.21.15.122:0/3665143777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4180af3e0 0x7fd4181467c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd4080af040 tx=0x7fd408019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:11.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd4157fa640 1 -- 172.21.15.122:0/3665143777 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4080b8670 con 0x7fd4180af3e0 2024-03-16T20:46:11.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd4157fa640 1 -- 172.21.15.122:0/3665143777 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4080b87d0 con 0x7fd4180af3e0 2024-03-16T20:46:11.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 -- 172.21.15.122:0/3665143777 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd41809a7b0 con 0x7fd4180af3e0 2024-03-16T20:46:11.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd4157fa640 1 -- 172.21.15.122:0/3665143777 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4080b8b80 con 0x7fd4180af3e0 2024-03-16T20:46:11.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.738+0000 7fd41eed0640 1 -- 172.21.15.122:0/3665143777 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41814e4f0 con 0x7fd4180af3e0 2024-03-16T20:46:11.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.742+0000 7fd41eed0640 1 -- 172.21.15.122:0/3665143777 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd41814e680 con 0x7fd4180af3e0 2024-03-16T20:46:11.751 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.742+0000 7fd4157fa640 1 -- 172.21.15.122:0/3665143777 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fd4080b8ce0 con 0x7fd4180af3e0 2024-03-16T20:46:11.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.742+0000 7fd4157fa640 1 --2- 172.21.15.122:0/3665143777 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd3f8066e40 0x7fd3f8069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.742+0000 7fd4157fa640 1 -- 172.21.15.122:0/3665143777 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd408120460 con 0x7fd4180af3e0 2024-03-16T20:46:11.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.742+0000 7fd4157fa640 1 --2- 172.21.15.122:0/3665143777 >> [v2:172.21.15.195:6810/1730924554,v1:172.21.15.195:6811/1730924554] conn(0x7fd3f806cd00 0x7fd3f806f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.742+0000 7fd4157fa640 1 -- 172.21.15.122:0/3665143777 --> [v2:172.21.15.195:6810/1730924554,v1:172.21.15.195:6811/1730924554] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd3f806f7d0 con 0x7fd3f806cd00 2024-03-16T20:46:11.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.742+0000 7fd4157fa640 1 -- 172.21.15.122:0/3665143777 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4081208b0 con 0x7fd4180af3e0 2024-03-16T20:46:11.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.742+0000 7fd41cc6c640 1 --2- 172.21.15.122:0/3665143777 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd3f8066e40 0x7fd3f8069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.742+0000 7fd417fff640 1 --2- 172.21.15.122:0/3665143777 >> [v2:172.21.15.195:6810/1730924554,v1:172.21.15.195:6811/1730924554] conn(0x7fd3f806cd00 0x7fd3f806f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.752 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.742+0000 7fd41cc6c640 1 --2- 172.21.15.122:0/3665143777 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd3f8066e40 0x7fd3f8069300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd41000b9d0 tx=0x7fd410008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:11.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.742+0000 7fd417fff640 1 --2- 172.21.15.122:0/3665143777 >> [v2:172.21.15.195:6810/1730924554,v1:172.21.15.195:6811/1730924554] conn(0x7fd3f806cd00 0x7fd3f806f120 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:11.753 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.746+0000 7fd4157fa640 1 -- 172.21.15.122:0/3665143777 <== osd.6 v2:172.21.15.195:6810/1730924554 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd3f806f7d0 con 0x7fd3f806cd00 2024-03-16T20:46:11.763 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.758+0000 7fd41eed0640 1 -- 172.21.15.122:0/3665143777 --> [v2:172.21.15.195:6810/1730924554,v1:172.21.15.195:6811/1730924554] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd418143e30 con 0x7fd3f806cd00 2024-03-16T20:46:11.764 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.758+0000 7fd4157fa640 1 -- 172.21.15.122:0/3665143777 <== osd.6 v2:172.21.15.195:6810/1730924554 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd418143e30 con 0x7fd3f806cd00 2024-03-16T20:46:11.764 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-16T20:46:11.765 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T20:46:11.765 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-16T20:46:11.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.758+0000 7fd41eed0640 1 -- 172.21.15.122:0/3665143777 >> [v2:172.21.15.195:6810/1730924554,v1:172.21.15.195:6811/1730924554] conn(0x7fd3f806cd00 msgr2=0x7fd3f806f120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.758+0000 7fd41eed0640 1 --2- 172.21.15.122:0/3665143777 >> [v2:172.21.15.195:6810/1730924554,v1:172.21.15.195:6811/1730924554] conn(0x7fd3f806cd00 0x7fd3f806f120 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.758+0000 7fd41eed0640 1 -- 172.21.15.122:0/3665143777 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd3f8066e40 msgr2=0x7fd3f8069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.758+0000 7fd41eed0640 1 --2- 172.21.15.122:0/3665143777 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fd3f8066e40 0x7fd3f8069300 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd41000b9d0 tx=0x7fd410008040 comp rx=0 tx=0).stop 2024-03-16T20:46:11.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.758+0000 7fd41eed0640 1 -- 172.21.15.122:0/3665143777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4180af3e0 msgr2=0x7fd4181467c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.758+0000 7fd41eed0640 1 --2- 172.21.15.122:0/3665143777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd4180af3e0 0x7fd4181467c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd4080af040 tx=0x7fd408019700 comp rx=0 tx=0).stop 2024-03-16T20:46:11.766 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.758+0000 7fd41d46d640 1 -- 172.21.15.122:0/3665143777 reap_dead start 2024-03-16T20:46:11.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.758+0000 7fd41eed0640 1 -- 172.21.15.122:0/3665143777 shutdown_connections 2024-03-16T20:46:11.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.758+0000 7fd41eed0640 1 -- 172.21.15.122:0/3665143777 >> 172.21.15.122:0/3665143777 conn(0x7fd4180b8d10 msgr2=0x7fd4180add40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:11.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.758+0000 7fd41eed0640 1 -- 172.21.15.122:0/3665143777 shutdown_connections 2024-03-16T20:46:11.767 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.758+0000 7fd41eed0640 1 -- 172.21.15.122:0/3665143777 wait complete. 2024-03-16T20:46:11.779 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-03-16T20:46:11.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a7aac640 1 Processor -- start 2024-03-16T20:46:11.895 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a7aac640 1 -- start start 2024-03-16T20:46:11.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a7aac640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7a0141580 0x7fe7a00baac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a7aac640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fe7a00bb000 0x7fe7a006cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a7aac640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe7a006d180 0x7fe7a014a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a7aac640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a0147ff0 con 0x7fe7a006d180 2024-03-16T20:46:11.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a7aac640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a00bb460 con 0x7fe7a00bb000 2024-03-16T20:46:11.897 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a7aac640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a01479a0 con 0x7fe7a0141580 2024-03-16T20:46:11.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a6049640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe7a006d180 0x7fe7a014a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a6049640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7fe7a006d180 0x7fe7a014a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54770/0 (socket says 172.21.15.122:54770) 2024-03-16T20:46:11.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a6049640 1 -- 172.21.15.122:0/3664388003 learned_addr learned my addr 172.21.15.122:0/3664388003 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:11.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a5047640 1 --2- 172.21.15.122:0/3664388003 >> v2:172.21.15.195:3300/0 conn(0x7fe7a00bb000 0x7fe7a006cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a5848640 1 --2- 172.21.15.122:0/3664388003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7a0141580 0x7fe7a00baac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a6049640 1 -- 172.21.15.122:0/3664388003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7a0141580 msgr2=0x7fe7a00baac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:11.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a6049640 1 --2- 172.21.15.122:0/3664388003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7a0141580 0x7fe7a00baac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.898 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a6049640 1 -- 172.21.15.122:0/3664388003 >> v2:172.21.15.195:3300/0 conn(0x7fe7a00bb000 msgr2=0x7fe7a006cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:11.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a6049640 1 --2- 172.21.15.122:0/3664388003 >> v2:172.21.15.195:3300/0 conn(0x7fe7a00bb000 0x7fe7a006cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.899 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.890+0000 7fe7a6049640 1 -- 172.21.15.122:0/3664388003 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a0147c80 con 0x7fe7a006d180 2024-03-16T20:46:11.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a6049640 1 --2- 172.21.15.122:0/3664388003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7a006d180 0x7fe7a014a5d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00a2c30 tx=0x7fe79801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a27260797c87963 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:11.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a4846640 1 -- 172.21.15.122:0/3664388003 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7980a5670 con 0x7fe7a006d180 2024-03-16T20:46:11.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a4846640 1 -- 172.21.15.122:0/3664388003 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7980a57d0 con 0x7fe7a006d180 2024-03-16T20:46:11.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a4846640 1 -- 172.21.15.122:0/3664388003 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7980a5b60 con 0x7fe7a006d180 2024-03-16T20:46:11.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a7aac640 1 -- 172.21.15.122:0/3664388003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7a006d180 msgr2=0x7fe7a014a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.900 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a7aac640 1 --2- 172.21.15.122:0/3664388003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7a006d180 0x7fe7a014a5d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00a2c30 tx=0x7fe79801c9f0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a7aac640 1 -- 172.21.15.122:0/3664388003 shutdown_connections 2024-03-16T20:46:11.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a7aac640 1 --2- 172.21.15.122:0/3664388003 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7a006d180 0x7fe7a014a5d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a7aac640 1 --2- 172.21.15.122:0/3664388003 >> v2:172.21.15.195:3300/0 conn(0x7fe7a00bb000 0x7fe7a006cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a7aac640 1 --2- 172.21.15.122:0/3664388003 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7a0141580 0x7fe7a00baac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a7aac640 1 -- 172.21.15.122:0/3664388003 >> 172.21.15.122:0/3664388003 conn(0x7fe7a00bca20 msgr2=0x7fe7a00bce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:11.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a7aac640 1 -- 172.21.15.122:0/3664388003 shutdown_connections 2024-03-16T20:46:11.901 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a7aac640 1 -- 172.21.15.122:0/3664388003 wait complete. 2024-03-16T20:46:11.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a7aac640 1 Processor -- start 2024-03-16T20:46:11.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a7aac640 1 -- start start 2024-03-16T20:46:11.903 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a7aac640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7a006d180 0x7fe7a0141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a7aac640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7a00bb000 0x7fe7a009fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a7aac640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7a0141580 0x7fe7a0144f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a7aac640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a0142290 con 0x7fe7a006d180 2024-03-16T20:46:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a7aac640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a01424a0 con 0x7fe7a0141580 2024-03-16T20:46:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a7aac640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a0142780 con 0x7fe7a00bb000 2024-03-16T20:46:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a5848640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7a006d180 0x7fe7a0141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a5848640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7a006d180 0x7fe7a0141d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54772/0 (socket says 172.21.15.122:54772) 2024-03-16T20:46:11.904 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a5848640 1 -- 172.21.15.122:0/934660040 learned_addr learned my addr 172.21.15.122:0/934660040 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a5047640 1 --2- 172.21.15.122:0/934660040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7a00bb000 0x7fe7a009fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a6049640 1 --2- 172.21.15.122:0/934660040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7a0141580 0x7fe7a0144f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a5848640 1 -- 172.21.15.122:0/934660040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7a00bb000 msgr2=0x7fe7a009fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a5848640 1 --2- 172.21.15.122:0/934660040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7a00bb000 0x7fe7a009fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a5848640 1 -- 172.21.15.122:0/934660040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7a0141580 msgr2=0x7fe7a0144f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a5848640 1 --2- 172.21.15.122:0/934660040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7a0141580 0x7fe7a0144f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.894+0000 7fe7a5848640 1 -- 172.21.15.122:0/934660040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe798009b30 con 0x7fe7a006d180 2024-03-16T20:46:11.905 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.898+0000 7fe7a5848640 1 --2- 172.21.15.122:0/934660040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7a006d180 0x7fe7a0141d50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00d800 tx=0x7fe79c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:11.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.898+0000 7fe7967fc640 1 -- 172.21.15.122:0/934660040 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c00b6c0 con 0x7fe7a006d180 2024-03-16T20:46:11.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.898+0000 7fe7967fc640 1 -- 172.21.15.122:0/934660040 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79c00b820 con 0x7fe7a006d180 2024-03-16T20:46:11.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.898+0000 7fe7a7aac640 1 -- 172.21.15.122:0/934660040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a00a1520 con 0x7fe7a006d180 2024-03-16T20:46:11.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.898+0000 7fe7967fc640 1 -- 172.21.15.122:0/934660040 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c00bb90 con 0x7fe7a006d180 2024-03-16T20:46:11.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.898+0000 7fe7a7aac640 1 -- 172.21.15.122:0/934660040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a00a1190 con 0x7fe7a006d180 2024-03-16T20:46:11.906 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.898+0000 7fe7a7aac640 1 -- 172.21.15.122:0/934660040 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a00a0540 con 0x7fe7a006d180 2024-03-16T20:46:11.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.898+0000 7fe7967fc640 1 -- 172.21.15.122:0/934660040 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fe79c00bcf0 con 0x7fe7a006d180 2024-03-16T20:46:11.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.898+0000 7fe7967fc640 1 --2- 172.21.15.122:0/934660040 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe784066e40 0x7fe784069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:11.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.898+0000 7fe7967fc640 1 -- 172.21.15.122:0/934660040 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe79c084bc0 con 0x7fe7a006d180 2024-03-16T20:46:11.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.898+0000 7fe7a5047640 1 --2- 172.21.15.122:0/934660040 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe784066e40 0x7fe784069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:11.907 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.898+0000 7fe7a5047640 1 --2- 172.21.15.122:0/934660040 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe784066e40 0x7fe784069300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe7a013af10 tx=0x7fe790008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:11.909 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:11.902+0000 7fe7967fc640 1 -- 172.21.15.122:0/934660040 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fe79c051d40 con 0x7fe7a006d180 2024-03-16T20:46:12.065 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.058+0000 7fe7a7aac640 1 -- 172.21.15.122:0/934660040 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7a00a13b0 con 0x7fe784066e40 2024-03-16T20:46:12.066 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.062+0000 7fe7967fc640 1 -- 172.21.15.122:0/934660040 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe7a00a13b0 con 0x7fe784066e40 2024-03-16T20:46:12.066 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 6 to deep-scrub 2024-03-16T20:46:12.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.062+0000 7fe7a7aac640 1 -- 172.21.15.122:0/934660040 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe784066e40 msgr2=0x7fe784069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.062+0000 7fe7a7aac640 1 --2- 172.21.15.122:0/934660040 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe784066e40 0x7fe784069300 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe7a013af10 tx=0x7fe790008040 comp rx=0 tx=0).stop 2024-03-16T20:46:12.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.062+0000 7fe7a7aac640 1 -- 172.21.15.122:0/934660040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7a006d180 msgr2=0x7fe7a0141d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.062+0000 7fe7a7aac640 1 --2- 172.21.15.122:0/934660040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7a006d180 0x7fe7a0141d50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00d800 tx=0x7fe79c00dcc0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.062+0000 7fe7a7aac640 1 -- 172.21.15.122:0/934660040 shutdown_connections 2024-03-16T20:46:12.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.062+0000 7fe7a7aac640 1 --2- 172.21.15.122:0/934660040 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fe784066e40 0x7fe784069300 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.062+0000 7fe7a7aac640 1 --2- 172.21.15.122:0/934660040 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fe7a0141580 0x7fe7a0144f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.069 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.062+0000 7fe7a7aac640 1 --2- 172.21.15.122:0/934660040 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fe7a00bb000 0x7fe7a009fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.062+0000 7fe7a7aac640 1 --2- 172.21.15.122:0/934660040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe7a006d180 0x7fe7a0141d50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.062+0000 7fe7a7aac640 1 -- 172.21.15.122:0/934660040 >> 172.21.15.122:0/934660040 conn(0x7fe7a00bca20 msgr2=0x7fe7a00b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:12.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.062+0000 7fe7a7aac640 1 -- 172.21.15.122:0/934660040 shutdown_connections 2024-03-16T20:46:12.070 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.062+0000 7fe7a7aac640 1 -- 172.21.15.122:0/934660040 wait complete. 2024-03-16T20:46:12.086 INFO:tasks.ceph:Scrubbing osd.7 2024-03-16T20:46:12.086 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-03-16T20:46:12.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fc74b9640 1 Processor -- start 2024-03-16T20:46:12.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fc74b9640 1 -- start start 2024-03-16T20:46:12.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fc74b9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc009abd0 0x7f7fc00b7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fc74b9640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc00b3e30 con 0x7f7fc009afb0 2024-03-16T20:46:12.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fc74b9640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc00b45a0 con 0x7f7fc009dd20 2024-03-16T20:46:12.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fc74b9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc00b7e60 con 0x7f7fc009abd0 2024-03-16T20:46:12.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fc5a56640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f7fc009afb0 0x7f7fc006d0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:42196/0 (socket says 172.21.15.122:42196) 2024-03-16T20:46:12.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fc5a56640 1 -- 172.21.15.122:0/141189464 learned_addr learned my addr 172.21.15.122:0/141189464 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:46:12.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fb7fff640 1 -- 172.21.15.122:0/141189464 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066542819 0 0) 0x7f7fc00b3e30 con 0x7f7fc009afb0 2024-03-16T20:46:12.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fc5255640 1 --2- 172.21.15.122:0/141189464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc009abd0 0x7f7fc00b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fb7fff640 1 -- 172.21.15.122:0/141189464 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fac003530 con 0x7f7fc009afb0 2024-03-16T20:46:12.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fb7fff640 1 -- 172.21.15.122:0/141189464 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3729735691 0 0) 0x7f7fac003530 con 0x7f7fc009afb0 2024-03-16T20:46:12.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fb7fff640 1 -- 172.21.15.122:0/141189464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc009abd0 msgr2=0x7f7fc00b7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:12.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fb7fff640 1 --2- 172.21.15.122:0/141189464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc009abd0 0x7f7fc00b7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fb7fff640 1 -- 172.21.15.122:0/141189464 >> v1:172.21.15.195:6789/0 conn(0x7f7fc009dd20 legacy=0x7f7fc006c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fb7fff640 1 -- 172.21.15.122:0/141189464 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc006d7e0 con 0x7f7fc009afb0 2024-03-16T20:46:12.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fb7fff640 1 -- 172.21.15.122:0/141189464 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f7fb8001a20 con 0x7f7fc009afb0 2024-03-16T20:46:12.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fb7fff640 1 -- 172.21.15.122:0/141189464 >> v1:172.21.15.122:6789/0 conn(0x7f7fc009afb0 legacy=0x7f7fc006d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fb7fff640 1 --2- 172.21.15.122:0/141189464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc009abd0 0x7f7fac0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fb7fff640 1 --2- 172.21.15.122:0/141189464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fac003fe0 0x7f7fac0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fb7fff640 1 --2- 172.21.15.122:0/141189464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fac004d10 0x7f7fac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fb7fff640 1 -- 172.21.15.122:0/141189464 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc00b7e60 con 0x7f7fac004d10 2024-03-16T20:46:12.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fb7fff640 1 -- 172.21.15.122:0/141189464 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac00bbd0 con 0x7f7fac003fe0 2024-03-16T20:46:12.211 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fb7fff640 1 -- 172.21.15.122:0/141189464 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac00bd90 con 0x7f7fc009abd0 2024-03-16T20:46:12.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fc5a56640 1 --2- 172.21.15.122:0/141189464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fac004d10 0x7f7fac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fc4a54640 1 --2- 172.21.15.122:0/141189464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fac003fe0 0x7f7fac0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fc5255640 1 --2- 172.21.15.122:0/141189464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc009abd0 0x7f7fac0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fc5a56640 1 -- 172.21.15.122:0/141189464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc009abd0 msgr2=0x7f7fac0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:12.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fc5a56640 1 --2- 172.21.15.122:0/141189464 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc009abd0 0x7f7fac0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fc5a56640 1 -- 172.21.15.122:0/141189464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fac003fe0 msgr2=0x7f7fac0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:12.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fc5a56640 1 --2- 172.21.15.122:0/141189464 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fac003fe0 0x7f7fac0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.212 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.202+0000 7f7fc5a56640 1 -- 172.21.15.122:0/141189464 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fac003530 con 0x7f7fac004d10 2024-03-16T20:46:12.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc5a56640 1 --2- 172.21.15.122:0/141189464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fac004d10 0x7f7fac005190 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7fc009b400 tx=0x7f7fb801fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b260fc2b3136fff server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:12.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fb7fff640 1 -- 172.21.15.122:0/141189464 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb80aa690 con 0x7f7fac004d10 2024-03-16T20:46:12.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fb7fff640 1 -- 172.21.15.122:0/141189464 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb80aa7f0 con 0x7f7fac004d10 2024-03-16T20:46:12.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fb7fff640 1 -- 172.21.15.122:0/141189464 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb80aaba0 con 0x7f7fac004d10 2024-03-16T20:46:12.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc74b9640 1 -- 172.21.15.122:0/141189464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fac004d10 msgr2=0x7f7fac005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc74b9640 1 --2- 172.21.15.122:0/141189464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fac004d10 0x7f7fac005190 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7fc009b400 tx=0x7f7fb801fa60 comp rx=0 tx=0).stop 2024-03-16T20:46:12.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc5a56640 1 -- 172.21.15.122:0/141189464 reap_dead start 2024-03-16T20:46:12.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc74b9640 1 -- 172.21.15.122:0/141189464 shutdown_connections 2024-03-16T20:46:12.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc74b9640 1 -- 172.21.15.122:0/141189464 >> 172.21.15.122:0/141189464 conn(0x7f7fc00b8d10 msgr2=0x7f7fc009a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:12.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc74b9640 1 -- 172.21.15.122:0/141189464 shutdown_connections 2024-03-16T20:46:12.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc74b9640 1 -- 172.21.15.122:0/141189464 wait complete. 2024-03-16T20:46:12.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc74b9640 1 Processor -- start 2024-03-16T20:46:12.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc74b9640 1 -- start start 2024-03-16T20:46:12.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc74b9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fc01ce850 0x7f7fc01df7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc74b9640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc00976b0 0x7f7fc0071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc74b9640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc01dfd10 0x7f7fc00718c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc74b9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc01ca0b0 con 0x7f7fc01ce850 2024-03-16T20:46:12.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc74b9640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc00726f0 con 0x7f7fc00976b0 2024-03-16T20:46:12.217 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc74b9640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc01cec30 con 0x7f7fc01dfd10 2024-03-16T20:46:12.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc5255640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fc01ce850 0x7f7fc01df7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc5255640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fc01ce850 0x7f7fc01df7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54788/0 (socket says 172.21.15.122:54788) 2024-03-16T20:46:12.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc5255640 1 -- 172.21.15.122:0/3464739077 learned_addr learned my addr 172.21.15.122:0/3464739077 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:12.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc4a54640 1 --2- 172.21.15.122:0/3464739077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc00976b0 0x7f7fc0071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc5a56640 1 --2- 172.21.15.122:0/3464739077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc01dfd10 0x7f7fc00718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc5255640 1 -- 172.21.15.122:0/3464739077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc01dfd10 msgr2=0x7f7fc00718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc5255640 1 --2- 172.21.15.122:0/3464739077 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f7fc01dfd10 0x7f7fc00718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc5255640 1 -- 172.21.15.122:0/3464739077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc00976b0 msgr2=0x7f7fc0071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.218 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc5255640 1 --2- 172.21.15.122:0/3464739077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc00976b0 0x7f7fc0071380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc5255640 1 -- 172.21.15.122:0/3464739077 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc01ca540 con 0x7f7fc01ce850 2024-03-16T20:46:12.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc4a54640 1 --2- 172.21.15.122:0/3464739077 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f7fc00976b0 0x7f7fc0071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:12.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.206+0000 7f7fc5255640 1 --2- 172.21.15.122:0/3464739077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fc01ce850 0x7f7fc01df7d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00e810 tx=0x7f7fbc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:12.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.210+0000 7f7fb5ffb640 1 -- 172.21.15.122:0/3464739077 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc00c6d0 con 0x7f7fc01ce850 2024-03-16T20:46:12.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.210+0000 7f7fb5ffb640 1 -- 172.21.15.122:0/3464739077 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fbc00c830 con 0x7f7fc01ce850 2024-03-16T20:46:12.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.210+0000 7f7fc74b9640 1 -- 172.21.15.122:0/3464739077 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc01cb2f0 con 0x7f7fc01ce850 2024-03-16T20:46:12.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.210+0000 7f7fb5ffb640 1 -- 172.21.15.122:0/3464739077 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc00cbe0 con 0x7f7fc01ce850 2024-03-16T20:46:12.219 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.210+0000 7f7fc74b9640 1 -- 172.21.15.122:0/3464739077 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc0072990 con 0x7f7fc01ce850 2024-03-16T20:46:12.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.210+0000 7f7fc74b9640 1 -- 172.21.15.122:0/3464739077 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7fc00b4810 con 0x7f7fc01ce850 2024-03-16T20:46:12.220 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.210+0000 7f7fb5ffb640 1 -- 172.21.15.122:0/3464739077 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f7fbc00cd40 con 0x7f7fc01ce850 2024-03-16T20:46:12.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.210+0000 7f7fb5ffb640 1 --2- 172.21.15.122:0/3464739077 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7fa4066e40 0x7f7fa4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.210+0000 7f7fb5ffb640 1 -- 172.21.15.122:0/3464739077 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7fbc0847f0 con 0x7f7fc01ce850 2024-03-16T20:46:12.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.210+0000 7f7fb5ffb640 1 --2- 172.21.15.122:0/3464739077 >> [v2:172.21.15.195:6824/3035467983,v1:172.21.15.195:6825/3035467983] conn(0x7f7fa406cd00 0x7f7fa406f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.210+0000 7f7fb5ffb640 1 -- 172.21.15.122:0/3464739077 --> [v2:172.21.15.195:6824/3035467983,v1:172.21.15.195:6825/3035467983] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fa406f7d0 con 0x7f7fa406cd00 2024-03-16T20:46:12.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.210+0000 7f7fb5ffb640 1 -- 172.21.15.122:0/3464739077 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fbc084c40 con 0x7f7fc01ce850 2024-03-16T20:46:12.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.210+0000 7f7fc4a54640 1 --2- 172.21.15.122:0/3464739077 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7fa4066e40 0x7f7fa4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.210+0000 7f7fc5a56640 1 --2- 172.21.15.122:0/3464739077 >> [v2:172.21.15.195:6824/3035467983,v1:172.21.15.195:6825/3035467983] conn(0x7f7fa406cd00 0x7f7fa406f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.210+0000 7f7fc5a56640 1 --2- 172.21.15.122:0/3464739077 >> [v2:172.21.15.195:6824/3035467983,v1:172.21.15.195:6825/3035467983] conn(0x7f7fa406cd00 0x7f7fa406f120 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:12.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.210+0000 7f7fc4a54640 1 --2- 172.21.15.122:0/3464739077 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7fa4066e40 0x7f7fa4069300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7fc01c9ef0 tx=0x7f7fb0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:12.222 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.214+0000 7f7fb5ffb640 1 -- 172.21.15.122:0/3464739077 <== osd.7 v2:172.21.15.195:6824/3035467983 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f7fa406f7d0 con 0x7f7fa406cd00 2024-03-16T20:46:12.231 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.226+0000 7f7fc74b9640 1 -- 172.21.15.122:0/3464739077 --> [v2:172.21.15.195:6824/3035467983,v1:172.21.15.195:6825/3035467983] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7fc0072050 con 0x7f7fa406cd00 2024-03-16T20:46:12.232 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.226+0000 7f7fb5ffb640 1 -- 172.21.15.122:0/3464739077 <== osd.7 v2:172.21.15.195:6824/3035467983 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f7fc0072050 con 0x7f7fa406cd00 2024-03-16T20:46:12.232 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-03-16T20:46:12.232 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-03-16T20:46:12.232 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-03-16T20:46:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.226+0000 7f7fc74b9640 1 -- 172.21.15.122:0/3464739077 >> [v2:172.21.15.195:6824/3035467983,v1:172.21.15.195:6825/3035467983] conn(0x7f7fa406cd00 msgr2=0x7f7fa406f120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.226+0000 7f7fc74b9640 1 --2- 172.21.15.122:0/3464739077 >> [v2:172.21.15.195:6824/3035467983,v1:172.21.15.195:6825/3035467983] conn(0x7f7fa406cd00 0x7f7fa406f120 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.226+0000 7f7fc74b9640 1 -- 172.21.15.122:0/3464739077 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7fa4066e40 msgr2=0x7f7fa4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.233 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.226+0000 7f7fc74b9640 1 --2- 172.21.15.122:0/3464739077 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f7fa4066e40 0x7f7fa4069300 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7fc01c9ef0 tx=0x7f7fb0009040 comp rx=0 tx=0).stop 2024-03-16T20:46:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.226+0000 7f7fc74b9640 1 -- 172.21.15.122:0/3464739077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fc01ce850 msgr2=0x7f7fc01df7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.226+0000 7f7fc74b9640 1 --2- 172.21.15.122:0/3464739077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fc01ce850 0x7f7fc01df7d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc00e810 tx=0x7f7fbc00ecd0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.226+0000 7f7fc5a56640 1 -- 172.21.15.122:0/3464739077 reap_dead start 2024-03-16T20:46:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.226+0000 7f7fc74b9640 1 -- 172.21.15.122:0/3464739077 shutdown_connections 2024-03-16T20:46:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.226+0000 7f7fc74b9640 1 -- 172.21.15.122:0/3464739077 >> 172.21.15.122:0/3464739077 conn(0x7f7fc00b8d10 msgr2=0x7f7fc006c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.226+0000 7f7fc74b9640 1 -- 172.21.15.122:0/3464739077 shutdown_connections 2024-03-16T20:46:12.234 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.226+0000 7f7fc74b9640 1 -- 172.21.15.122:0/3464739077 wait complete. 2024-03-16T20:46:12.247 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-03-16T20:46:12.359 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.354+0000 7fae6bdcf640 1 Processor -- start 2024-03-16T20:46:12.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.354+0000 7fae6bdcf640 1 -- start start 2024-03-16T20:46:12.360 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.354+0000 7fae6bdcf640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fae640adf80 0x7fae64072510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.354+0000 7fae6bdcf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae6414b1b0 0x7fae64072a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.354+0000 7fae6bdcf640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae640aec70 con 0x7fae64066fa0 2024-03-16T20:46:12.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.354+0000 7fae6bdcf640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae64072f90 con 0x7fae640adf80 2024-03-16T20:46:12.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.354+0000 7fae6bdcf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae640ae690 con 0x7fae6414b1b0 2024-03-16T20:46:12.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.354+0000 7fae6a36c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae6414b1b0 0x7fae64072a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.354+0000 7fae6936a640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7fae640adf80 0x7fae64072510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.354+0000 7fae6a36c640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae6414b1b0 0x7fae64072a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3301/0 says I am v2:172.21.15.122:51590/0 (socket says 172.21.15.122:51590) 2024-03-16T20:46:12.361 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.354+0000 7fae6a36c640 1 -- 172.21.15.122:0/1378532147 learned_addr learned my addr 172.21.15.122:0/1378532147 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:12.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.354+0000 7fae6a36c640 1 -- 172.21.15.122:0/1378532147 >> v2:172.21.15.195:3300/0 conn(0x7fae640adf80 msgr2=0x7fae64072510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:12.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.354+0000 7fae68b69640 1 -- 172.21.15.122:0/1378532147 <== mon.1 v1:172.21.15.195:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 611686183 0 0) 0x7fae640aec70 con 0x7fae64066fa0 2024-03-16T20:46:12.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.354+0000 7fae6a36c640 1 --2- 172.21.15.122:0/1378532147 >> v2:172.21.15.195:3300/0 conn(0x7fae640adf80 0x7fae64072510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.354+0000 7fae6a36c640 1 -- 172.21.15.122:0/1378532147 >> v1:172.21.15.195:6789/0 conn(0x7fae64066fa0 legacy=0x7fae6414aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.362 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.354+0000 7fae6a36c640 1 -- 172.21.15.122:0/1378532147 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae64067380 con 0x7fae6414b1b0 2024-03-16T20:46:12.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.358+0000 7fae6a36c640 1 --2- 172.21.15.122:0/1378532147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae6414b1b0 0x7fae64072a50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fae54009870 tx=0x7fae5401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6398f21d0e1c81ca server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:12.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.358+0000 7fae68b69640 1 -- 172.21.15.122:0/1378532147 <== mon.2 v2:172.21.15.195:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae540a6040 con 0x7fae6414b1b0 2024-03-16T20:46:12.364 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.358+0000 7fae68b69640 1 -- 172.21.15.122:0/1378532147 <== mon.2 v2:172.21.15.195:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae540aa600 con 0x7fae6414b1b0 2024-03-16T20:46:12.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.358+0000 7fae68b69640 1 -- 172.21.15.122:0/1378532147 <== mon.2 v2:172.21.15.195:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae540aa990 con 0x7fae6414b1b0 2024-03-16T20:46:12.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.358+0000 7fae6bdcf640 1 -- 172.21.15.122:0/1378532147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae6414b1b0 msgr2=0x7fae64072a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.365 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.358+0000 7fae6bdcf640 1 --2- 172.21.15.122:0/1378532147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae6414b1b0 0x7fae64072a50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fae54009870 tx=0x7fae5401c4a0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.358+0000 7fae6bdcf640 1 -- 172.21.15.122:0/1378532147 shutdown_connections 2024-03-16T20:46:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.358+0000 7fae6bdcf640 1 --2- 172.21.15.122:0/1378532147 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae6414b1b0 0x7fae64072a50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.358+0000 7fae6bdcf640 1 --2- 172.21.15.122:0/1378532147 >> v2:172.21.15.195:3300/0 conn(0x7fae640adf80 0x7fae64072510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.358+0000 7fae6bdcf640 1 -- 172.21.15.122:0/1378532147 >> 172.21.15.122:0/1378532147 conn(0x7fae6406d2b0 msgr2=0x7fae6406d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.358+0000 7fae6bdcf640 1 -- 172.21.15.122:0/1378532147 shutdown_connections 2024-03-16T20:46:12.366 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.358+0000 7fae6bdcf640 1 -- 172.21.15.122:0/1378532147 wait complete. 2024-03-16T20:46:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6bdcf640 1 Processor -- start 2024-03-16T20:46:12.368 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6bdcf640 1 -- start start 2024-03-16T20:46:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6bdcf640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae64066fa0 0x7fae641df410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6bdcf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae640adf80 0x7fae6409de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6bdcf640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae6414b1b0 0x7fae641e3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6bdcf640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae641e2950 con 0x7fae640adf80 2024-03-16T20:46:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6bdcf640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae641e38a0 con 0x7fae6414b1b0 2024-03-16T20:46:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6bdcf640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae641e2ed0 con 0x7fae64066fa0 2024-03-16T20:46:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6936a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae640adf80 0x7fae6409de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6936a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae640adf80 0x7fae6409de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54792/0 (socket says 172.21.15.122:54792) 2024-03-16T20:46:12.369 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6936a640 1 -- 172.21.15.122:0/3390737665 learned_addr learned my addr 172.21.15.122:0/3390737665 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6a36c640 1 --2- 172.21.15.122:0/3390737665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae6414b1b0 0x7fae641e3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae69b6b640 1 --2- 172.21.15.122:0/3390737665 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae64066fa0 0x7fae641df410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6936a640 1 -- 172.21.15.122:0/3390737665 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae64066fa0 msgr2=0x7fae641df410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6936a640 1 --2- 172.21.15.122:0/3390737665 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae64066fa0 0x7fae641df410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6936a640 1 -- 172.21.15.122:0/3390737665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae6414b1b0 msgr2=0x7fae641e3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6936a640 1 --2- 172.21.15.122:0/3390737665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae6414b1b0 0x7fae641e3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.370 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6936a640 1 -- 172.21.15.122:0/3390737665 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae540094d0 con 0x7fae640adf80 2024-03-16T20:46:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae69b6b640 1 --2- 172.21.15.122:0/3390737665 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae64066fa0 0x7fae641df410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6936a640 1 --2- 172.21.15.122:0/3390737665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae640adf80 0x7fae6409de20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fae6000e960 tx=0x7fae6000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae5a7fc640 1 -- 172.21.15.122:0/3390737665 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae60009690 con 0x7fae640adf80 2024-03-16T20:46:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6bdcf640 1 -- 172.21.15.122:0/3390737665 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae64142250 con 0x7fae640adf80 2024-03-16T20:46:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae5a7fc640 1 -- 172.21.15.122:0/3390737665 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae6000ca00 con 0x7fae640adf80 2024-03-16T20:46:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae5a7fc640 1 -- 172.21.15.122:0/3390737665 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae6000cd70 con 0x7fae640adf80 2024-03-16T20:46:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6bdcf640 1 -- 172.21.15.122:0/3390737665 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae641424c0 con 0x7fae640adf80 2024-03-16T20:46:12.371 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae6bdcf640 1 -- 172.21.15.122:0/3390737665 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae641e3080 con 0x7fae640adf80 2024-03-16T20:46:12.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae5a7fc640 1 -- 172.21.15.122:0/3390737665 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7fae60013470 con 0x7fae640adf80 2024-03-16T20:46:12.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae5a7fc640 1 --2- 172.21.15.122:0/3390737665 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fae48066e70 0x7fae48069330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae5a7fc640 1 -- 172.21.15.122:0/3390737665 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fae600849c0 con 0x7fae640adf80 2024-03-16T20:46:12.372 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.362+0000 7fae69b6b640 1 --2- 172.21.15.122:0/3390737665 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fae48066e70 0x7fae48069330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.373 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.366+0000 7fae69b6b640 1 --2- 172.21.15.122:0/3390737665 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fae48066e70 0x7fae48069330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fae5c007fd0 tx=0x7fae5c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:12.374 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.370+0000 7fae5a7fc640 1 -- 172.21.15.122:0/3390737665 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7fae641e3080 con 0x7fae640adf80 2024-03-16T20:46:12.522 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.518+0000 7fae6bdcf640 1 -- 172.21.15.122:0/3390737665 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fae641e3040 con 0x7fae48066e70 2024-03-16T20:46:12.523 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.518+0000 7fae5a7fc640 1 -- 172.21.15.122:0/3390737665 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fae641e3040 con 0x7fae48066e70 2024-03-16T20:46:12.524 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 7 to deep-scrub 2024-03-16T20:46:12.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.518+0000 7fae6bdcf640 1 -- 172.21.15.122:0/3390737665 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fae48066e70 msgr2=0x7fae48069330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.518+0000 7fae6bdcf640 1 --2- 172.21.15.122:0/3390737665 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fae48066e70 0x7fae48069330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fae5c007fd0 tx=0x7fae5c009040 comp rx=0 tx=0).stop 2024-03-16T20:46:12.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.518+0000 7fae6bdcf640 1 -- 172.21.15.122:0/3390737665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae640adf80 msgr2=0x7fae6409de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.518+0000 7fae6bdcf640 1 --2- 172.21.15.122:0/3390737665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae640adf80 0x7fae6409de20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fae6000e960 tx=0x7fae6000ee20 comp rx=0 tx=0).stop 2024-03-16T20:46:12.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.518+0000 7fae6bdcf640 1 -- 172.21.15.122:0/3390737665 shutdown_connections 2024-03-16T20:46:12.526 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.518+0000 7fae6bdcf640 1 --2- 172.21.15.122:0/3390737665 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7fae48066e70 0x7fae48069330 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.518+0000 7fae6bdcf640 1 --2- 172.21.15.122:0/3390737665 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7fae6414b1b0 0x7fae641e3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.518+0000 7fae6bdcf640 1 --2- 172.21.15.122:0/3390737665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fae640adf80 0x7fae6409de20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.518+0000 7fae6bdcf640 1 --2- 172.21.15.122:0/3390737665 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7fae64066fa0 0x7fae641df410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.518+0000 7fae6bdcf640 1 -- 172.21.15.122:0/3390737665 >> 172.21.15.122:0/3390737665 conn(0x7fae6406d2b0 msgr2=0x7fae640b74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:12.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.518+0000 7fae6bdcf640 1 -- 172.21.15.122:0/3390737665 shutdown_connections 2024-03-16T20:46:12.527 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.518+0000 7fae6bdcf640 1 -- 172.21.15.122:0/3390737665 wait complete. 2024-03-16T20:46:12.541 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T20:46:12.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.638+0000 7f11304e4640 1 Processor -- start 2024-03-16T20:46:12.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.638+0000 7f11304e4640 1 -- start start 2024-03-16T20:46:12.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.642+0000 7f11304e4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f1128072120 0x7f11280afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.642+0000 7f11304e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112813ed60 0x7f112813f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.647 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.642+0000 7f11304e4640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1128148430 0x7f11281488c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.642+0000 7f11304e4640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11280b1050 con 0x7f1128148430 2024-03-16T20:46:12.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.642+0000 7f11304e4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1128072500 con 0x7f1128072120 2024-03-16T20:46:12.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.642+0000 7f11304e4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11280b38a0 con 0x7f112813ed60 2024-03-16T20:46:12.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.642+0000 7f112ea81640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1128148430 0x7f11281488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.642+0000 7f112ea81640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f1128148430 0x7f11281488c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54806/0 (socket says 172.21.15.122:54806) 2024-03-16T20:46:12.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.642+0000 7f112ea81640 1 -- 172.21.15.122:0/1354513875 learned_addr learned my addr 172.21.15.122:0/1354513875 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:12.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.642+0000 7f112da7f640 1 --2- 172.21.15.122:0/1354513875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112813ed60 0x7f112813f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.642+0000 7f112e280640 1 --2- 172.21.15.122:0/1354513875 >> v2:172.21.15.195:3300/0 conn(0x7f1128072120 0x7f11280afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.648 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.642+0000 7f112ea81640 1 -- 172.21.15.122:0/1354513875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112813ed60 msgr2=0x7f112813f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:12.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.642+0000 7f112ea81640 1 --2- 172.21.15.122:0/1354513875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112813ed60 0x7f112813f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.642+0000 7f112ea81640 1 -- 172.21.15.122:0/1354513875 >> v2:172.21.15.195:3300/0 conn(0x7f1128072120 msgr2=0x7f11280afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:12.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.642+0000 7f112ea81640 1 --2- 172.21.15.122:0/1354513875 >> v2:172.21.15.195:3300/0 conn(0x7f1128072120 0x7f11280afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.649 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.642+0000 7f112ea81640 1 -- 172.21.15.122:0/1354513875 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11280b3b80 con 0x7f1128148430 2024-03-16T20:46:12.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f112ea81640 1 --2- 172.21.15.122:0/1354513875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1128148430 0x7f11281488c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1128072790 tx=0x7f112001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26539be2dcf0a195 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:12.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f112d27e640 1 -- 172.21.15.122:0/1354513875 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11200a55e0 con 0x7f1128148430 2024-03-16T20:46:12.650 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f112d27e640 1 -- 172.21.15.122:0/1354513875 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11200a5740 con 0x7f1128148430 2024-03-16T20:46:12.651 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f112d27e640 1 -- 172.21.15.122:0/1354513875 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11200a5ad0 con 0x7f1128148430 2024-03-16T20:46:12.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 -- 172.21.15.122:0/1354513875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1128148430 msgr2=0x7f11281488c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.653 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 --2- 172.21.15.122:0/1354513875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1128148430 0x7f11281488c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1128072790 tx=0x7f112001c960 comp rx=0 tx=0).stop 2024-03-16T20:46:12.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 -- 172.21.15.122:0/1354513875 shutdown_connections 2024-03-16T20:46:12.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 --2- 172.21.15.122:0/1354513875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1128148430 0x7f11281488c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 --2- 172.21.15.122:0/1354513875 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112813ed60 0x7f112813f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 --2- 172.21.15.122:0/1354513875 >> v2:172.21.15.195:3300/0 conn(0x7f1128072120 0x7f11280afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 -- 172.21.15.122:0/1354513875 >> 172.21.15.122:0/1354513875 conn(0x7f11280b8860 msgr2=0x7f11280b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:12.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 -- 172.21.15.122:0/1354513875 shutdown_connections 2024-03-16T20:46:12.654 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 -- 172.21.15.122:0/1354513875 wait complete. 2024-03-16T20:46:12.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 Processor -- start 2024-03-16T20:46:12.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 -- start start 2024-03-16T20:46:12.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1128072120 0x7f11280b7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112813ed60 0x7f1128071520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.657 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1128148430 0x7f1128070100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1128071340 con 0x7f1128072120 2024-03-16T20:46:12.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f112814ef00 con 0x7f1128148430 2024-03-16T20:46:12.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11280710a0 con 0x7f112813ed60 2024-03-16T20:46:12.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f112e280640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1128072120 0x7f11280b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f112e280640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1128072120 0x7f11280b7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:54820/0 (socket says 172.21.15.122:54820) 2024-03-16T20:46:12.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f112e280640 1 -- 172.21.15.122:0/3538582133 learned_addr learned my addr 172.21.15.122:0/3538582133 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:12.658 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f112da7f640 1 --2- 172.21.15.122:0/3538582133 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112813ed60 0x7f1128071520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f112ea81640 1 --2- 172.21.15.122:0/3538582133 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1128148430 0x7f1128070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f112e280640 1 -- 172.21.15.122:0/3538582133 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112813ed60 msgr2=0x7f1128071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f112e280640 1 --2- 172.21.15.122:0/3538582133 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112813ed60 0x7f1128071520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f112e280640 1 -- 172.21.15.122:0/3538582133 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1128148430 msgr2=0x7f1128070100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f112e280640 1 --2- 172.21.15.122:0/3538582133 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1128148430 0x7f1128070100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f112e280640 1 -- 172.21.15.122:0/3538582133 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1120009b30 con 0x7f1128072120 2024-03-16T20:46:12.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f112e280640 1 --2- 172.21.15.122:0/3538582133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1128072120 0x7f11280b7740 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f112400d800 tx=0x7f112400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:12.659 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f111effd640 1 -- 172.21.15.122:0/3538582133 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11240096a0 con 0x7f1128072120 2024-03-16T20:46:12.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f111effd640 1 -- 172.21.15.122:0/3538582133 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1124009800 con 0x7f1128072120 2024-03-16T20:46:12.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 -- 172.21.15.122:0/3538582133 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1128143030 con 0x7f1128072120 2024-03-16T20:46:12.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f111effd640 1 -- 172.21.15.122:0/3538582133 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1124009b70 con 0x7f1128072120 2024-03-16T20:46:12.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.646+0000 7f11304e4640 1 -- 172.21.15.122:0/3538582133 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11280b8020 con 0x7f1128072120 2024-03-16T20:46:12.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.650+0000 7f11304e4640 1 -- 172.21.15.122:0/3538582133 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1128075280 con 0x7f1128072120 2024-03-16T20:46:12.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.650+0000 7f111effd640 1 -- 172.21.15.122:0/3538582133 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f1124009cd0 con 0x7f1128072120 2024-03-16T20:46:12.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.650+0000 7f111effd640 1 --2- 172.21.15.122:0/3538582133 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f110c066e40 0x7f110c069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:12.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.650+0000 7f111effd640 1 -- 172.21.15.122:0/3538582133 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1124084e70 con 0x7f1128072120 2024-03-16T20:46:12.660 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.650+0000 7f112da7f640 1 --2- 172.21.15.122:0/3538582133 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f110c066e40 0x7f110c069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:12.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.650+0000 7f112da7f640 1 --2- 172.21.15.122:0/3538582133 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f110c066e40 0x7f110c069300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f11281459b0 tx=0x7f1118008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:12.661 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.654+0000 7f111effd640 1 -- 172.21.15.122:0/3538582133 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1124051ff0 con 0x7f1128072120 2024-03-16T20:46:12.804 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.798+0000 7f11304e4640 1 -- 172.21.15.122:0/3538582133 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f112813d3a0 con 0x7f110c066e40 2024-03-16T20:46:12.806 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.798+0000 7f111effd640 1 -- 172.21.15.122:0/3538582133 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13383 (secure 0 0 0) 0x7f112813d3a0 con 0x7f110c066e40 2024-03-16T20:46:12.806 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:46:12.807 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-03-16T20:46:12.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.802+0000 7f11304e4640 1 -- 172.21.15.122:0/3538582133 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f110c066e40 msgr2=0x7f110c069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.802+0000 7f11304e4640 1 --2- 172.21.15.122:0/3538582133 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f110c066e40 0x7f110c069300 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f11281459b0 tx=0x7f1118008040 comp rx=0 tx=0).stop 2024-03-16T20:46:12.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.802+0000 7f11304e4640 1 -- 172.21.15.122:0/3538582133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1128072120 msgr2=0x7f11280b7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:12.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.802+0000 7f11304e4640 1 --2- 172.21.15.122:0/3538582133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1128072120 0x7f11280b7740 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f112400d800 tx=0x7f112400dcc0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.802+0000 7f11304e4640 1 -- 172.21.15.122:0/3538582133 shutdown_connections 2024-03-16T20:46:12.808 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.802+0000 7f11304e4640 1 --2- 172.21.15.122:0/3538582133 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f110c066e40 0x7f110c069300 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.802+0000 7f11304e4640 1 --2- 172.21.15.122:0/3538582133 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1128148430 0x7f1128070100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.802+0000 7f11304e4640 1 --2- 172.21.15.122:0/3538582133 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f112813ed60 0x7f1128071520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.802+0000 7f11304e4640 1 --2- 172.21.15.122:0/3538582133 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1128072120 0x7f11280b7740 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:12.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.802+0000 7f11304e4640 1 -- 172.21.15.122:0/3538582133 >> 172.21.15.122:0/3538582133 conn(0x7f11280b8860 msgr2=0x7f11280b58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:12.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.802+0000 7f11304e4640 1 -- 172.21.15.122:0/3538582133 shutdown_connections 2024-03-16T20:46:12.809 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:12.802+0000 7f11304e4640 1 -- 172.21.15.122:0/3538582133 wait complete. 2024-03-16T20:46:12.822 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-03-16T20:46:12.599725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236948,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737772,"statfs":{"total":773094113280,"available":772851478528,"internally_reserved":0,"allocated":4235264,"data_stored":2351392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.569170"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-16T20:46:05.378168+0000","last_change":"2024-03-16T20:44:54.293418+0000","last_active":"2024-03-16T20:46:05.378168+0000","last_peered":"2024-03-16T20:46:05.378168+0000","last_clean":"2024-03-16T20:46:05.378168+0000","last_became_active":"2024-03-16T20:44:54.291347+0000","last_became_peered":"2024-03-16T20:44:54.291347+0000","last_unstale":"2024-03-16T20:46:05.378168+0000","last_undegraded":"2024-03-16T20:46:05.378168+0000","last_fullsized":"2024-03-16T20:46:05.378168+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-16T20:44:53.077639+0000","last_clean_scrub_stamp":"2024-03-16T20:44:53.077639+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-18T03:29:18.318494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":974848,"data_stored":736700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31556,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31556,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343868,"statfs":{"total":96636764160,"available":96608120832,"internally_reserved":0,"allocated":974848,"data_stored":736700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T20:46:12.825 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-03-16T20:44:53.077639+0000 time.struct_time(tm_year=2024, tm_mon=3, tm_mday=16, tm_hour=20, tm_min=44, tm_sec=53, tm_wday=5, tm_yday=76, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=3, tm_mday=16, tm_hour=20, tm_min=46, tm_sec=7, tm_wday=5, tm_yday=76, tm_isdst=0) 2024-03-16T20:46:12.825 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-03-16T20:46:32.826 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-16T20:46:32.957 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.950+0000 7f3992687640 1 Processor -- start 2024-03-16T20:46:32.958 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.950+0000 7f3992687640 1 -- start start 2024-03-16T20:46:32.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f3992687640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f398c147e90 0x7f398c1471e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:32.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f3992687640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f398c0ba850 0x7f398c09b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:32.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f3992687640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398c09aee0 con 0x7f398c09ba00 2024-03-16T20:46:32.959 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f3992687640 1 -- --> v2:172.21.15.122:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f398c0bb1d0 con 0x7f398c147e90 2024-03-16T20:46:32.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f3992687640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f398c0b9280 con 0x7f398c0ba850 2024-03-16T20:46:32.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f398bfff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f398c147e90 0x7f398c1471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:32.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f398bfff640 1 --2- >> v2:172.21.15.122:3300/0 conn(0x7f398c147e90 0x7f398c1471e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48488/0 (socket says 172.21.15.122:48488) 2024-03-16T20:46:32.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f398bfff640 1 -- 172.21.15.122:0/1559729234 learned_addr learned my addr 172.21.15.122:0/1559729234 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:32.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f398b7fe640 1 --2- 172.21.15.122:0/1559729234 >> v2:172.21.15.195:3300/0 conn(0x7f398c0ba850 0x7f398c09b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:32.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f398bfff640 1 -- 172.21.15.122:0/1559729234 >> v2:172.21.15.195:3300/0 conn(0x7f398c0ba850 msgr2=0x7f398c09b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:32.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f398bfff640 1 --2- 172.21.15.122:0/1559729234 >> v2:172.21.15.195:3300/0 conn(0x7f398c0ba850 0x7f398c09b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:32.960 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f398bfff640 1 -- 172.21.15.122:0/1559729234 >> v1:172.21.15.195:6789/0 conn(0x7f398c09ba00 legacy=0x7f398c0b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:32.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f398bfff640 1 -- 172.21.15.122:0/1559729234 --> v2:172.21.15.122:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c09ab70 con 0x7f398c147e90 2024-03-16T20:46:32.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f398b7fe640 1 --2- 172.21.15.122:0/1559729234 >> v2:172.21.15.195:3300/0 conn(0x7f398c0ba850 0x7f398c09b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:32.961 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f398bfff640 1 --2- 172.21.15.122:0/1559729234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c147e90 0x7f398c1471e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f398c06d5a0 tx=0x7f397c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd82d011f84d524 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:32.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f398affd640 1 -- 172.21.15.122:0/1559729234 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c0ab600 con 0x7f398c147e90 2024-03-16T20:46:32.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f398affd640 1 -- 172.21.15.122:0/1559729234 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c0a7050 con 0x7f398c147e90 2024-03-16T20:46:32.962 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.954+0000 7f398affd640 1 -- 172.21.15.122:0/1559729234 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c0ab980 con 0x7f398c147e90 2024-03-16T20:46:32.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 -- 172.21.15.122:0/1559729234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c147e90 msgr2=0x7f398c1471e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:32.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 --2- 172.21.15.122:0/1559729234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c147e90 0x7f398c1471e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f398c06d5a0 tx=0x7f397c01c3c0 comp rx=0 tx=0).stop 2024-03-16T20:46:32.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 -- 172.21.15.122:0/1559729234 shutdown_connections 2024-03-16T20:46:32.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 --2- 172.21.15.122:0/1559729234 >> v2:172.21.15.195:3300/0 conn(0x7f398c0ba850 0x7f398c09b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:32.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 --2- 172.21.15.122:0/1559729234 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c147e90 0x7f398c1471e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:32.963 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 -- 172.21.15.122:0/1559729234 >> 172.21.15.122:0/1559729234 conn(0x7f398c0bca50 msgr2=0x7f398c0bce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:32.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 -- 172.21.15.122:0/1559729234 shutdown_connections 2024-03-16T20:46:32.964 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 -- 172.21.15.122:0/1559729234 wait complete. 2024-03-16T20:46:32.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 Processor -- start 2024-03-16T20:46:32.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 -- start start 2024-03-16T20:46:32.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f398c09ba00 0x7f398c13f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:32.966 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f398c147e90 0x7f398c1dfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:32.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c1413d0 0x7f398c1e0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:32.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c13a5f0 con 0x7f398c1413d0 2024-03-16T20:46:32.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c13a840 con 0x7f398c147e90 2024-03-16T20:46:32.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c13ab20 con 0x7f398c09ba00 2024-03-16T20:46:32.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3990c24640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c1413d0 0x7f398c1e0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:32.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3990c24640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c1413d0 0x7f398c1e0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48496/0 (socket says 172.21.15.122:48496) 2024-03-16T20:46:32.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3990c24640 1 -- 172.21.15.122:0/2275264392 learned_addr learned my addr 172.21.15.122:0/2275264392 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:32.967 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f398bfff640 1 --2- 172.21.15.122:0/2275264392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f398c09ba00 0x7f398c13f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:32.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f398b7fe640 1 --2- 172.21.15.122:0/2275264392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f398c147e90 0x7f398c1dfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:32.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3990c24640 1 -- 172.21.15.122:0/2275264392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f398c09ba00 msgr2=0x7f398c13f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:32.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3990c24640 1 --2- 172.21.15.122:0/2275264392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f398c09ba00 0x7f398c13f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:32.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3990c24640 1 -- 172.21.15.122:0/2275264392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f398c147e90 msgr2=0x7f398c1dfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:32.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3990c24640 1 --2- 172.21.15.122:0/2275264392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f398c147e90 0x7f398c1dfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:32.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3990c24640 1 -- 172.21.15.122:0/2275264392 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c0094d0 con 0x7f398c1413d0 2024-03-16T20:46:32.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f398bfff640 1 --2- 172.21.15.122:0/2275264392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f398c09ba00 0x7f398c13f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:32.968 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3990c24640 1 --2- 172.21.15.122:0/2275264392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c1413d0 0x7f398c1e0130 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f398400b970 tx=0x7f398400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:32.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3988ff9640 1 -- 172.21.15.122:0/2275264392 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398400cb40 con 0x7f398c1413d0 2024-03-16T20:46:32.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3988ff9640 1 -- 172.21.15.122:0/2275264392 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398400cca0 con 0x7f398c1413d0 2024-03-16T20:46:32.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 -- 172.21.15.122:0/2275264392 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398c1df6a0 con 0x7f398c1413d0 2024-03-16T20:46:32.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3988ff9640 1 -- 172.21.15.122:0/2275264392 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984014680 con 0x7f398c1413d0 2024-03-16T20:46:32.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.958+0000 7f3992687640 1 -- 172.21.15.122:0/2275264392 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c1396e0 con 0x7f398c1413d0 2024-03-16T20:46:32.969 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.962+0000 7f3992687640 1 -- 172.21.15.122:0/2275264392 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398c1df910 con 0x7f398c1413d0 2024-03-16T20:46:32.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.962+0000 7f3988ff9640 1 -- 172.21.15.122:0/2275264392 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f398401d3b0 con 0x7f398c1413d0 2024-03-16T20:46:32.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.962+0000 7f3988ff9640 1 --2- 172.21.15.122:0/2275264392 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3978066e40 0x7f3978069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:32.970 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.962+0000 7f3988ff9640 1 -- 172.21.15.122:0/2275264392 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3984084610 con 0x7f398c1413d0 2024-03-16T20:46:32.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.962+0000 7f398bfff640 1 --2- 172.21.15.122:0/2275264392 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3978066e40 0x7f3978069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:32.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.962+0000 7f398bfff640 1 --2- 172.21.15.122:0/2275264392 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3978066e40 0x7f3978069300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f397c0ad000 tx=0x7f397c008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:32.971 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:32.966+0000 7f3988ff9640 1 -- 172.21.15.122:0/2275264392 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f398c1df910 con 0x7f398c1413d0 2024-03-16T20:46:33.126 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.122+0000 7f3992687640 1 -- 172.21.15.122:0/2275264392 --> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f398c075280 con 0x7f3978066e40 2024-03-16T20:46:33.128 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.122+0000 7f3988ff9640 1 -- 172.21.15.122:0/2275264392 <== mgr.4105 v2:172.21.15.195:6832/3072163791 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13354 (secure 0 0 0) 0x7f398c075280 con 0x7f3978066e40 2024-03-16T20:46:33.128 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:46:33.129 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-03-16T20:46:33.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.126+0000 7f3992687640 1 -- 172.21.15.122:0/2275264392 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3978066e40 msgr2=0x7f3978069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:33.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.126+0000 7f3992687640 1 --2- 172.21.15.122:0/2275264392 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3978066e40 0x7f3978069300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f397c0ad000 tx=0x7f397c008040 comp rx=0 tx=0).stop 2024-03-16T20:46:33.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.126+0000 7f3992687640 1 -- 172.21.15.122:0/2275264392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c1413d0 msgr2=0x7f398c1e0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:33.131 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.126+0000 7f3992687640 1 --2- 172.21.15.122:0/2275264392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c1413d0 0x7f398c1e0130 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f398400b970 tx=0x7f398400be30 comp rx=0 tx=0).stop 2024-03-16T20:46:33.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.126+0000 7f3992687640 1 -- 172.21.15.122:0/2275264392 shutdown_connections 2024-03-16T20:46:33.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.126+0000 7f3992687640 1 --2- 172.21.15.122:0/2275264392 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f3978066e40 0x7f3978069300 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.126+0000 7f3992687640 1 --2- 172.21.15.122:0/2275264392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f398c1413d0 0x7f398c1e0130 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.126+0000 7f3992687640 1 --2- 172.21.15.122:0/2275264392 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f398c147e90 0x7f398c1dfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.126+0000 7f3992687640 1 --2- 172.21.15.122:0/2275264392 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f398c09ba00 0x7f398c13f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.126+0000 7f3992687640 1 -- 172.21.15.122:0/2275264392 >> 172.21.15.122:0/2275264392 conn(0x7f398c0bca50 msgr2=0x7f398c0bce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:33.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.126+0000 7f3992687640 1 -- 172.21.15.122:0/2275264392 shutdown_connections 2024-03-16T20:46:33.132 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.126+0000 7f3992687640 1 -- 172.21.15.122:0/2275264392 wait complete. 2024-03-16T20:46:33.147 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-03-16T20:46:32.604892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":236944,"kb_used_data":4136,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737776,"statfs":{"total":773094113280,"available":772851482624,"internally_reserved":0,"allocated":4235264,"data_stored":2351392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003071"},"pg_stats":[{"pgid":"1.0","version":"33'119","reported_seq":204,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-03-16T20:46:12.649283+0000","last_change":"2024-03-16T20:46:12.649348+0000","last_active":"2024-03-16T20:46:12.649283+0000","last_peered":"2024-03-16T20:46:12.649283+0000","last_clean":"2024-03-16T20:46:12.649283+0000","last_became_active":"2024-03-16T20:44:54.291347+0000","last_became_peered":"2024-03-16T20:44:54.291347+0000","last_unstale":"2024-03-16T20:46:12.649283+0000","last_undegraded":"2024-03-16T20:46:12.649283+0000","last_fullsized":"2024-03-16T20:46:12.649283+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'119","last_scrub_stamp":"2024-03-16T20:46:12.649100+0000","last_deep_scrub":"33'119","last_deep_scrub_stamp":"2024-03-16T20:46:12.649100+0000","last_clean_scrub_stamp":"2024-03-16T20:46:12.649100+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-03-17T20:57:24.711639+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32128,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339712,"statfs":{"total":96636764160,"available":96603865088,"internally_reserved":0,"allocated":974848,"data_stored":736700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31556,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31552,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31556,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340284,"statfs":{"total":96636764160,"available":96604450816,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27392,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344448,"statfs":{"total":96636764160,"available":96608714752,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27396,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":380928,"data_stored":146332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343872,"statfs":{"total":96636764160,"available":96608124928,"internally_reserved":0,"allocated":974848,"data_stored":736700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-16T20:46:33.148 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-03-16T20:46:33.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e079b4640 1 Processor -- start 2024-03-16T20:46:33.270 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e079b4640 1 -- start start 2024-03-16T20:46:33.273 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e079b4640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f2e0009ae30 0x7f2e000b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:33.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e079b4640 1 -- --> v1:172.21.15.122:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e000b3fa0 con 0x7f2e000b6510 2024-03-16T20:46:33.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e079b4640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0009b210 con 0x7f2e000b82f0 2024-03-16T20:46:33.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e079b4640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e000b77c0 con 0x7f2e0009ae30 2024-03-16T20:46:33.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e04f4f640 1 --1- >> v1:172.21.15.122:6789/0 conn(0x7f2e000b6510 0x7f2e000b7be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.122:6789/0 says I am v1:172.21.15.122:43324/0 (socket says 172.21.15.122:43324) 2024-03-16T20:46:33.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e04f4f640 1 -- 172.21.15.122:0/529126737 learned_addr learned my addr 172.21.15.122:0/529126737 (peer_addr_for_me v1:172.21.15.122:0/0) 2024-03-16T20:46:33.274 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2df3fff640 1 -- 172.21.15.122:0/529126737 <== mon.0 v1:172.21.15.122:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552425661 0 0) 0x7f2e000b3fa0 con 0x7f2e000b6510 2024-03-16T20:46:33.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e05750640 1 --2- 172.21.15.122:0/529126737 >> v2:172.21.15.195:3300/0 conn(0x7f2e0009ae30 0x7f2e000b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:33.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2df3fff640 1 -- 172.21.15.122:0/529126737 --> v1:172.21.15.122:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2dec003530 con 0x7f2e000b6510 2024-03-16T20:46:33.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2df3fff640 1 -- 172.21.15.122:0/529126737 <== mon.0 v1:172.21.15.122:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 339202940 0 0) 0x7f2dec003530 con 0x7f2e000b6510 2024-03-16T20:46:33.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2df3fff640 1 -- 172.21.15.122:0/529126737 >> v2:172.21.15.195:3300/0 conn(0x7f2e0009ae30 msgr2=0x7f2e000b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:33.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2df3fff640 1 --2- 172.21.15.122:0/529126737 >> v2:172.21.15.195:3300/0 conn(0x7f2e0009ae30 0x7f2e000b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2df3fff640 1 -- 172.21.15.122:0/529126737 >> v1:172.21.15.195:6789/0 conn(0x7f2e000b82f0 legacy=0x7f2e00142270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:33.275 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2df3fff640 1 -- 172.21.15.122:0/529126737 --> v1:172.21.15.122:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e000b4b20 con 0x7f2e000b6510 2024-03-16T20:46:33.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2df3fff640 1 -- 172.21.15.122:0/529126737 <== mon.0 v1:172.21.15.122:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2529622803 0 0) 0x7f2dfc001680 con 0x7f2e000b6510 2024-03-16T20:46:33.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2df3fff640 1 -- 172.21.15.122:0/529126737 >> v1:172.21.15.122:6789/0 conn(0x7f2e000b6510 legacy=0x7f2e000b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:33.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2df3fff640 1 --2- 172.21.15.122:0/529126737 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dec003980 0x7f2dec003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:33.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2df3fff640 1 --2- 172.21.15.122:0/529126737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dec0047d0 0x7f2dec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:33.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2df3fff640 1 --2- 172.21.15.122:0/529126737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dec005190 0x7f2dec00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:33.276 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2df3fff640 1 -- 172.21.15.122:0/529126737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e000b77c0 con 0x7f2dec005190 2024-03-16T20:46:33.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2df3fff640 1 -- 172.21.15.122:0/529126737 --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec00c030 con 0x7f2dec003980 2024-03-16T20:46:33.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2df3fff640 1 -- 172.21.15.122:0/529126737 --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec00c1f0 con 0x7f2dec0047d0 2024-03-16T20:46:33.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e05f51640 1 --2- 172.21.15.122:0/529126737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dec005190 0x7f2dec00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:33.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e04f4f640 1 --2- 172.21.15.122:0/529126737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dec0047d0 0x7f2dec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:33.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e05750640 1 --2- 172.21.15.122:0/529126737 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dec003980 0x7f2dec003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:33.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e05f51640 1 -- 172.21.15.122:0/529126737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dec0047d0 msgr2=0x7f2dec004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:33.277 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e05f51640 1 --2- 172.21.15.122:0/529126737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dec0047d0 0x7f2dec004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e05f51640 1 -- 172.21.15.122:0/529126737 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dec003980 msgr2=0x7f2dec003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:33.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e05f51640 1 --2- 172.21.15.122:0/529126737 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2dec003980 0x7f2dec003da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e05f51640 1 -- 172.21.15.122:0/529126737 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dec003530 con 0x7f2dec005190 2024-03-16T20:46:33.278 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.266+0000 7f2e04f4f640 1 --2- 172.21.15.122:0/529126737 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2dec0047d0 0x7f2dec004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:33.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e05f51640 1 -- 172.21.15.122:0/529126737 reap_dead start 2024-03-16T20:46:33.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e05f51640 1 --2- 172.21.15.122:0/529126737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dec005190 0x7f2dec00b990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2df8001e70 tx=0x7f2df801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1be19a44815296 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:33.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2df3fff640 1 -- 172.21.15.122:0/529126737 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8003b80 con 0x7f2dec005190 2024-03-16T20:46:33.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2df3fff640 1 -- 172.21.15.122:0/529126737 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df8003d00 con 0x7f2dec005190 2024-03-16T20:46:33.283 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2df3fff640 1 -- 172.21.15.122:0/529126737 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8005260 con 0x7f2dec005190 2024-03-16T20:46:33.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e079b4640 1 -- 172.21.15.122:0/529126737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dec005190 msgr2=0x7f2dec00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:33.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e079b4640 1 --2- 172.21.15.122:0/529126737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dec005190 0x7f2dec00b990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2df8001e70 tx=0x7f2df801c780 comp rx=0 tx=0).stop 2024-03-16T20:46:33.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e079b4640 1 -- 172.21.15.122:0/529126737 shutdown_connections 2024-03-16T20:46:33.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e079b4640 1 --2- 172.21.15.122:0/529126737 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dec005190 0x7f2dec00b990 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.284 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e079b4640 1 -- 172.21.15.122:0/529126737 >> 172.21.15.122:0/529126737 conn(0x7f2e000b8d10 msgr2=0x7f2e0009a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:33.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e079b4640 1 -- 172.21.15.122:0/529126737 shutdown_connections 2024-03-16T20:46:33.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e079b4640 1 -- 172.21.15.122:0/529126737 wait complete. 2024-03-16T20:46:33.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e079b4640 1 Processor -- start 2024-03-16T20:46:33.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e079b4640 1 -- start start 2024-03-16T20:46:33.285 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e079b4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dec005190 0x7f2e00140750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:33.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e079b4640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e000afef0 0x7f2e000b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:33.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e079b4640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e001431c0 0x7f2e00143620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:33.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e079b4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e000b08b0 con 0x7f2dec005190 2024-03-16T20:46:33.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e079b4640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e000b1070 con 0x7f2e001431c0 2024-03-16T20:46:33.286 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e079b4640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e000b1350 con 0x7f2e000afef0 2024-03-16T20:46:33.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e05750640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dec005190 0x7f2e00140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:33.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e05750640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dec005190 0x7f2e00140750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48524/0 (socket says 172.21.15.122:48524) 2024-03-16T20:46:33.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e05750640 1 -- 172.21.15.122:0/79109840 learned_addr learned my addr 172.21.15.122:0/79109840 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:33.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e05f51640 1 --2- 172.21.15.122:0/79109840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e001431c0 0x7f2e00143620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:33.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e05750640 1 -- 172.21.15.122:0/79109840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e000afef0 msgr2=0x7f2e000b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:33.287 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e05750640 1 --2- 172.21.15.122:0/79109840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e000afef0 0x7f2e000b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e05750640 1 -- 172.21.15.122:0/79109840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e001431c0 msgr2=0x7f2e00143620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:33.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e05750640 1 --2- 172.21.15.122:0/79109840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e001431c0 0x7f2e00143620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e05750640 1 -- 172.21.15.122:0/79109840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e000b1630 con 0x7f2dec005190 2024-03-16T20:46:33.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e05750640 1 --2- 172.21.15.122:0/79109840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dec005190 0x7f2e00140750 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2df4010890 tx=0x7f2df4010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:33.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2df1ffb640 1 -- 172.21.15.122:0/79109840 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df4009630 con 0x7f2dec005190 2024-03-16T20:46:33.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2df1ffb640 1 -- 172.21.15.122:0/79109840 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df400eb60 con 0x7f2dec005190 2024-03-16T20:46:33.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e079b4640 1 -- 172.21.15.122:0/79109840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e000b2a10 con 0x7f2dec005190 2024-03-16T20:46:33.288 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2df1ffb640 1 -- 172.21.15.122:0/79109840 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df401c400 con 0x7f2dec005190 2024-03-16T20:46:33.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.270+0000 7f2e079b4640 1 -- 172.21.15.122:0/79109840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e00072d40 con 0x7f2dec005190 2024-03-16T20:46:33.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.274+0000 7f2e079b4640 1 -- 172.21.15.122:0/79109840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e00074bb0 con 0x7f2dec005190 2024-03-16T20:46:33.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.274+0000 7f2df1ffb640 1 -- 172.21.15.122:0/79109840 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f2df401c560 con 0x7f2dec005190 2024-03-16T20:46:33.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.274+0000 7f2df1ffb640 1 --2- 172.21.15.122:0/79109840 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2de4066e40 0x7f2de4069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:33.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.274+0000 7f2df1ffb640 1 -- 172.21.15.122:0/79109840 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2df4084d90 con 0x7f2dec005190 2024-03-16T20:46:33.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.274+0000 7f2e04f4f640 1 --2- 172.21.15.122:0/79109840 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2de4066e40 0x7f2de4069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:33.289 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.274+0000 7f2e04f4f640 1 --2- 172.21.15.122:0/79109840 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2de4066e40 0x7f2de4069300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc000c10 tx=0x7f2dfc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:33.290 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.278+0000 7f2df1ffb640 1 -- 172.21.15.122:0/79109840 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f2df4051f20 con 0x7f2dec005190 2024-03-16T20:46:33.432 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.426+0000 7f2e079b4640 1 -- 172.21.15.122:0/79109840 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f2e000b17c0 con 0x7f2dec005190 2024-03-16T20:46:33.438 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.430+0000 7f2df1ffb640 1 -- 172.21.15.122:0/79109840 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f2df40555e0 con 0x7f2dec005190 2024-03-16T20:46:33.439 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.434+0000 7f2df1ffb640 1 -- 172.21.15.122:0/79109840 <== mon.0 v2:172.21.15.122:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2df4051f20 con 0x7f2dec005190 2024-03-16T20:46:33.441 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.434+0000 7f2e079b4640 1 -- 172.21.15.122:0/79109840 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2de4066e40 msgr2=0x7f2de4069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:33.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.434+0000 7f2e079b4640 1 --2- 172.21.15.122:0/79109840 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2de4066e40 0x7f2de4069300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc000c10 tx=0x7f2dfc009040 comp rx=0 tx=0).stop 2024-03-16T20:46:33.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.434+0000 7f2e079b4640 1 -- 172.21.15.122:0/79109840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dec005190 msgr2=0x7f2e00140750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:33.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.434+0000 7f2e079b4640 1 --2- 172.21.15.122:0/79109840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dec005190 0x7f2e00140750 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2df4010890 tx=0x7f2df4010d50 comp rx=0 tx=0).stop 2024-03-16T20:46:33.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.434+0000 7f2e079b4640 1 -- 172.21.15.122:0/79109840 shutdown_connections 2024-03-16T20:46:33.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.434+0000 7f2e079b4640 1 --2- 172.21.15.122:0/79109840 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f2de4066e40 0x7f2de4069300 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.442 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.434+0000 7f2e079b4640 1 --2- 172.21.15.122:0/79109840 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f2e001431c0 0x7f2e00143620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.434+0000 7f2e079b4640 1 --2- 172.21.15.122:0/79109840 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f2e000afef0 0x7f2e000b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.434+0000 7f2e079b4640 1 --2- 172.21.15.122:0/79109840 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2dec005190 0x7f2e00140750 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.434+0000 7f2e079b4640 1 -- 172.21.15.122:0/79109840 >> 172.21.15.122:0/79109840 conn(0x7f2e000b8d10 msgr2=0x7f2e001d00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.434+0000 7f2e079b4640 1 -- 172.21.15.122:0/79109840 shutdown_connections 2024-03-16T20:46:33.443 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.434+0000 7f2e079b4640 1 -- 172.21.15.122:0/79109840 wait complete. 2024-03-16T20:46:33.456 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-16T20:46:33.457 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-16T20:46:33.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.566+0000 7f140602b640 1 Processor -- start 2024-03-16T20:46:33.574 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.566+0000 7f140602b640 1 -- start start 2024-03-16T20:46:33.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.570+0000 7f140602b640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f14000b2690 0x7f14000b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:33.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.570+0000 7f140602b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1400071a20 0x7f14001445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:33.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.570+0000 7f140602b640 1 -- --> v1:172.21.15.195:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14000b46e0 con 0x7f1400143dd0 2024-03-16T20:46:33.575 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.570+0000 7f140602b640 1 -- --> v2:172.21.15.195:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f140013e5e0 con 0x7f14000b2690 2024-03-16T20:46:33.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.570+0000 7f140602b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14000b2a70 con 0x7f1400071a20 2024-03-16T20:46:33.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.570+0000 7f13fffff640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1400071a20 0x7f14001445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:33.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.570+0000 7f13ff7fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f14000b2690 0x7f14000b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:33.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.570+0000 7f13ff7fe640 1 --2- >> v2:172.21.15.195:3300/0 conn(0x7f14000b2690 0x7f14000b41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.195:3300/0 says I am v2:172.21.15.122:52328/0 (socket says 172.21.15.122:52328) 2024-03-16T20:46:33.576 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.570+0000 7f13ff7fe640 1 -- 172.21.15.122:0/3858982982 learned_addr learned my addr 172.21.15.122:0/3858982982 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:33.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.570+0000 7f13ff7fe640 1 -- 172.21.15.122:0/3858982982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1400071a20 msgr2=0x7f14001445a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-16T20:46:33.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.570+0000 7f13ff7fe640 1 --2- 172.21.15.122:0/3858982982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1400071a20 0x7f14001445a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.570+0000 7f13ff7fe640 1 -- 172.21.15.122:0/3858982982 >> v1:172.21.15.195:6789/0 conn(0x7f1400143dd0 legacy=0x7f1400147be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:33.577 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.570+0000 7f13ff7fe640 1 -- 172.21.15.122:0/3858982982 --> v2:172.21.15.195:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14000b4950 con 0x7f14000b2690 2024-03-16T20:46:33.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.570+0000 7f13fffff640 1 --2- 172.21.15.122:0/3858982982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1400071a20 0x7f14001445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-16T20:46:33.578 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13ff7fe640 1 --2- 172.21.15.122:0/3858982982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14000b2690 0x7f14000b41a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009870 tx=0x7f13f401c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34d3b2611979ec86 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:33.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13fe7fc640 1 -- 172.21.15.122:0/3858982982 <== mon.1 v2:172.21.15.195:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4004230 con 0x7f14000b2690 2024-03-16T20:46:33.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13fe7fc640 1 -- 172.21.15.122:0/3858982982 <== mon.1 v2:172.21.15.195:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f13f40a7050 con 0x7f14000b2690 2024-03-16T20:46:33.579 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13fe7fc640 1 -- 172.21.15.122:0/3858982982 <== mon.1 v2:172.21.15.195:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4003620 con 0x7f14000b2690 2024-03-16T20:46:33.582 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 -- 172.21.15.122:0/3858982982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14000b2690 msgr2=0x7f14000b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:33.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 --2- 172.21.15.122:0/3858982982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14000b2690 0x7f14000b41a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009870 tx=0x7f13f401c3f0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 -- 172.21.15.122:0/3858982982 shutdown_connections 2024-03-16T20:46:33.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 --2- 172.21.15.122:0/3858982982 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1400071a20 0x7f14001445a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 --2- 172.21.15.122:0/3858982982 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f14000b2690 0x7f14000b41a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 -- 172.21.15.122:0/3858982982 >> 172.21.15.122:0/3858982982 conn(0x7f14000b8860 msgr2=0x7f14000b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:33.583 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 -- 172.21.15.122:0/3858982982 shutdown_connections 2024-03-16T20:46:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 -- 172.21.15.122:0/3858982982 wait complete. 2024-03-16T20:46:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 Processor -- start 2024-03-16T20:46:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 -- start start 2024-03-16T20:46:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 --2- >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1400071a20 0x7f14001cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14000b2690 0x7f140013a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 --2- >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1400143dd0 0x7f14000b6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:33.584 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14000b7b30 con 0x7f14000b2690 2024-03-16T20:46:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 -- --> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14000b7df0 con 0x7f1400143dd0 2024-03-16T20:46:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 -- --> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1400144f30 con 0x7f1400071a20 2024-03-16T20:46:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13feffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14000b2690 0x7f140013a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13feffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14000b2690 0x7f140013a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48532/0 (socket says 172.21.15.122:48532) 2024-03-16T20:46:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13feffd640 1 -- 172.21.15.122:0/3085587872 learned_addr learned my addr 172.21.15.122:0/3085587872 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-03-16T20:46:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13ff7fe640 1 --2- 172.21.15.122:0/3085587872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1400071a20 0x7f14001cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:33.585 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13fffff640 1 --2- 172.21.15.122:0/3085587872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1400143dd0 0x7f14000b6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13feffd640 1 -- 172.21.15.122:0/3085587872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1400071a20 msgr2=0x7f14001cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13feffd640 1 --2- 172.21.15.122:0/3085587872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1400071a20 0x7f14001cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13feffd640 1 -- 172.21.15.122:0/3085587872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1400143dd0 msgr2=0x7f14000b6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13feffd640 1 --2- 172.21.15.122:0/3085587872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1400143dd0 0x7f14000b6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13feffd640 1 -- 172.21.15.122:0/3085587872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f800bc80 con 0x7f14000b2690 2024-03-16T20:46:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13fffff640 1 --2- 172.21.15.122:0/3085587872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1400143dd0 0x7f14000b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-16T20:46:33.586 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13feffd640 1 --2- 172.21.15.122:0/3085587872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14000b2690 0x7f140013a1a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009e00 tx=0x7f13f000c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:33.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13e7fff640 1 -- 172.21.15.122:0/3085587872 <== mon.0 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f000cdb0 con 0x7f14000b2690 2024-03-16T20:46:33.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13e7fff640 1 -- 172.21.15.122:0/3085587872 <== mon.0 v2:172.21.15.122:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f13f0004020 con 0x7f14000b2690 2024-03-16T20:46:33.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 -- 172.21.15.122:0/3085587872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f40094d0 con 0x7f14000b2690 2024-03-16T20:46:33.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f13e7fff640 1 -- 172.21.15.122:0/3085587872 <== mon.0 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0004470 con 0x7f14000b2690 2024-03-16T20:46:33.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.574+0000 7f140602b640 1 -- 172.21.15.122:0/3085587872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1400145510 con 0x7f14000b2690 2024-03-16T20:46:33.587 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.578+0000 7f140602b640 1 -- 172.21.15.122:0/3085587872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1400146b20 con 0x7f14000b2690 2024-03-16T20:46:33.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.578+0000 7f13e7fff640 1 -- 172.21.15.122:0/3085587872 <== mon.0 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 5) v1 ==== 86409+0+0 (secure 0 0 0) 0x7f13f0003600 con 0x7f14000b2690 2024-03-16T20:46:33.588 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.578+0000 7f13e7fff640 1 --2- 172.21.15.122:0/3085587872 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f13ec066e40 0x7f13ec069300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-16T20:46:33.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.578+0000 7f13e7fff640 1 -- 172.21.15.122:0/3085587872 <== mon.0 v2:172.21.15.122:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f13f0086820 con 0x7f14000b2690 2024-03-16T20:46:33.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.578+0000 7f13ff7fe640 1 --2- 172.21.15.122:0/3085587872 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f13ec066e40 0x7f13ec069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-16T20:46:33.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.578+0000 7f13ff7fe640 1 --2- 172.21.15.122:0/3085587872 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f13ec066e40 0x7f13ec069300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f13f40ad000 tx=0x7f13f4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-16T20:46:33.589 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.582+0000 7f13e7fff640 1 -- 172.21.15.122:0/3085587872 <== mon.0 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176632 (secure 0 0 0) 0x7f1400146b20 con 0x7f14000b2690 2024-03-16T20:46:33.782 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.778+0000 7f140602b640 1 -- 172.21.15.122:0/3085587872 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1400146b20 con 0x7f14000b2690 2024-03-16T20:46:33.783 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.778+0000 7f13e7fff640 1 -- 172.21.15.122:0/3085587872 <== mon.0 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+2120 (secure 0 0 0) 0x7f1400146b20 con 0x7f14000b2690 2024-03-16T20:46:33.783 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-03-16T20:46:33.783 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":15,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4616,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.195:6835/3638578490","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.195:6834","nonce":3638578490},{"type":"v1","addr":"172.21.15.195:6835","nonce":3638578490}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5337,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/915029434","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":915029434},{"type":"v1","addr":"172.21.15.122:6833","nonce":915029434}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":15}],"filesystems":[]} 2024-03-16T20:46:33.784 INFO:teuthology.orchestra.run.smithi122.stderr:dumped fsmap epoch 15 2024-03-16T20:46:33.785 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.778+0000 7f140602b640 1 -- 172.21.15.122:0/3085587872 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f13ec066e40 msgr2=0x7f13ec069300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:33.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.778+0000 7f140602b640 1 --2- 172.21.15.122:0/3085587872 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f13ec066e40 0x7f13ec069300 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f13f40ad000 tx=0x7f13f4009040 comp rx=0 tx=0).stop 2024-03-16T20:46:33.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.778+0000 7f140602b640 1 -- 172.21.15.122:0/3085587872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14000b2690 msgr2=0x7f140013a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-16T20:46:33.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.778+0000 7f140602b640 1 --2- 172.21.15.122:0/3085587872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14000b2690 0x7f140013a1a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009e00 tx=0x7f13f000c430 comp rx=0 tx=0).stop 2024-03-16T20:46:33.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.778+0000 7f140602b640 1 -- 172.21.15.122:0/3085587872 shutdown_connections 2024-03-16T20:46:33.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.778+0000 7f140602b640 1 --2- 172.21.15.122:0/3085587872 >> [v2:172.21.15.195:6832/3072163791,v1:172.21.15.195:6833/3072163791] conn(0x7f13ec066e40 0x7f13ec069300 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.778+0000 7f140602b640 1 --2- 172.21.15.122:0/3085587872 >> [v2:172.21.15.195:3300/0,v1:172.21.15.195:6789/0] conn(0x7f1400143dd0 0x7f14000b6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.786 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.778+0000 7f140602b640 1 --2- 172.21.15.122:0/3085587872 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f14000b2690 0x7f140013a1a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.778+0000 7f140602b640 1 --2- 172.21.15.122:0/3085587872 >> [v2:172.21.15.195:3301/0,v1:172.21.15.195:6790/0] conn(0x7f1400071a20 0x7f14001cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-16T20:46:33.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.778+0000 7f140602b640 1 -- 172.21.15.122:0/3085587872 >> 172.21.15.122:0/3085587872 conn(0x7f14000b8860 msgr2=0x7f140006af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-16T20:46:33.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.778+0000 7f140602b640 1 -- 172.21.15.122:0/3085587872 shutdown_connections 2024-03-16T20:46:33.787 INFO:teuthology.orchestra.run.smithi122.stderr:2024-03-16T20:46:33.778+0000 7f140602b640 1 -- 172.21.15.122:0/3085587872 wait complete. 2024-03-16T20:46:33.801 DEBUG:tasks.cephfs.filesystem:already dead... 2024-03-16T20:46:33.801 INFO:teuthology.misc:Shutting down mds daemons... 2024-03-16T20:46:33.802 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-03-16T20:46:33.802 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:46:33.811 INFO:tasks.ceph.mds.a:Stopped 2024-03-16T20:46:33.811 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-03-16T20:46:33.811 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:46:33.829 INFO:tasks.ceph.mds.b:Stopped 2024-03-16T20:46:33.829 INFO:teuthology.misc:Shutting down osd daemons... 2024-03-16T20:46:33.829 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-03-16T20:46:33.829 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:46:36.736 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T20:46:36.739 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T20:46:39.931 INFO:tasks.ceph.osd.0:Stopped 2024-03-16T20:46:39.932 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-03-16T20:46:39.932 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:46:40.013 INFO:tasks.ceph.osd.1:Stopped 2024-03-16T20:46:40.013 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-03-16T20:46:40.014 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:46:40.097 INFO:tasks.ceph.osd.2:Stopped 2024-03-16T20:46:40.097 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-03-16T20:46:40.097 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:46:40.196 INFO:tasks.ceph.osd.3:Stopped 2024-03-16T20:46:40.197 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-03-16T20:46:40.197 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:46:40.269 INFO:tasks.ceph.osd.4:Stopped 2024-03-16T20:46:40.269 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-03-16T20:46:40.269 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:46:40.338 INFO:tasks.ceph.osd.5:Stopped 2024-03-16T20:46:40.339 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-03-16T20:46:40.339 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:46:40.409 INFO:tasks.ceph.osd.6:Stopped 2024-03-16T20:46:40.409 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-03-16T20:46:40.409 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:46:40.506 INFO:tasks.ceph.osd.7:Stopped 2024-03-16T20:46:40.507 INFO:teuthology.misc:Shutting down mgr daemons... 2024-03-16T20:46:40.507 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-03-16T20:46:40.507 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:46:40.535 INFO:tasks.ceph.mgr.y:Stopped 2024-03-16T20:46:40.535 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-03-16T20:46:40.535 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:46:40.564 INFO:tasks.ceph.mgr.x:Stopped 2024-03-16T20:46:40.564 INFO:teuthology.misc:Shutting down mon daemons... 2024-03-16T20:46:40.565 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-03-16T20:46:40.565 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:46:40.594 INFO:tasks.ceph.mon.a:Stopped 2024-03-16T20:46:40.595 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-03-16T20:46:40.595 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:46:40.624 INFO:tasks.ceph.mon.b:Stopped 2024-03-16T20:46:40.624 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-03-16T20:46:40.624 INFO:teuthology.orchestra.run:waiting for 300 2024-03-16T20:46:40.677 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-03-16T20:46:40.678 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-03-16T20:46:40.678 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-03-16T20:46:40.678 INFO:tasks.ceph.mon.c:Stopped 2024-03-16T20:46:40.678 INFO:tasks.ceph:Checking cluster log for badness... 2024-03-16T20:46:40.679 DEBUG:teuthology.orchestra.run.smithi122:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-03-16T20:46:40.695 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi122.front.sepia.ceph.com 2024-03-16T20:46:40.695 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-03-16T20:46:40.909 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi122.front.sepia.ceph.com 2024-03-16T20:46:40.909 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-03-16T20:46:41.129 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi122.front.sepia.ceph.com 2024-03-16T20:46:41.130 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-03-16T20:46:41.304 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi122.front.sepia.ceph.com 2024-03-16T20:46:41.305 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-03-16T20:46:41.497 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi195.front.sepia.ceph.com 2024-03-16T20:46:41.497 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-03-16T20:46:41.701 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi195.front.sepia.ceph.com 2024-03-16T20:46:41.701 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-03-16T20:46:41.966 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi195.front.sepia.ceph.com 2024-03-16T20:46:41.967 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-03-16T20:46:42.156 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi195.front.sepia.ceph.com 2024-03-16T20:46:42.157 DEBUG:teuthology.orchestra.run.smithi195:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-03-16T20:46:42.333 INFO:tasks.ceph:Archiving mon data... 2024-03-16T20:46:42.335 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602163/data/mon.a.tgz 2024-03-16T20:46:42.335 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-03-16T20:46:42.340 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.tnEI3SmXTL 2024-03-16T20:46:42.341 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.tnEI3SmXTL 2024-03-16T20:46:42.451 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.tnEI3SmXTL 2024-03-16T20:46:42.519 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.tnEI3SmXTL is 266KB 2024-03-16T20:46:42.576 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.tnEI3SmXTL 2024-03-16T20:46:42.581 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602163/data/mon.b.tgz 2024-03-16T20:46:42.581 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-03-16T20:46:42.587 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.6YNwxanxXI 2024-03-16T20:46:42.587 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.6YNwxanxXI 2024-03-16T20:46:42.715 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.6YNwxanxXI 2024-03-16T20:46:42.777 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.6YNwxanxXI is 358KB 2024-03-16T20:46:42.839 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.6YNwxanxXI 2024-03-16T20:46:42.844 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602163/data/mon.c.tgz 2024-03-16T20:46:42.844 DEBUG:teuthology.orchestra.run.smithi195:> mktemp 2024-03-16T20:46:42.889 INFO:teuthology.orchestra.run.smithi195.stdout:/tmp/tmp.SzsuPh2jsy 2024-03-16T20:46:42.890 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.SzsuPh2jsy 2024-03-16T20:46:43.020 DEBUG:teuthology.orchestra.run.smithi195:> sudo chmod 0666 /tmp/tmp.SzsuPh2jsy 2024-03-16T20:46:43.088 DEBUG:teuthology.orchestra.remote:smithi195:/tmp/tmp.SzsuPh2jsy is 358KB 2024-03-16T20:46:43.147 DEBUG:teuthology.orchestra.run.smithi195:> rm -fr /tmp/tmp.SzsuPh2jsy 2024-03-16T20:46:43.152 INFO:tasks.ceph:Cleaning ceph cluster... 2024-03-16T20:46:43.152 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-16T20:46:43.154 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-03-16T20:46:43.215 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-16T20:46:43.253 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-03-16T20:46:43.253 INFO:tasks.ceph:Archiving crash dumps... 2024-03-16T20:46:43.255 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602163/remote/smithi122/crash 2024-03-16T20:46:43.255 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-16T20:46:43.271 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/lib/ceph/crash to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602163/remote/smithi195/crash 2024-03-16T20:46:43.272 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-03-16T20:46:43.309 INFO:tasks.ceph:Shutting down logrotate 2024-03-16T20:46:43.309 DEBUG:teuthology.orchestra.run.smithi122:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T20:46:43.315 DEBUG:teuthology.orchestra.run.smithi195:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-03-16T20:46:43.388 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-16T20:46:43.405 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-03-16T20:46:43.445 INFO:tasks.ceph:Compressing logs... 2024-03-16T20:46:43.445 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T20:46:43.448 DEBUG:teuthology.orchestra.run.smithi195:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-16T20:46:43.467 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34242.log 2024-03-16T20:46:43.468 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34681.log 2024-03-16T20:46:43.468 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34242.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-03-16T20:46:43.470 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36712.log 2024-03-16T20:46:43.471 INFO:teuthology.orchestra.run.smithi122.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34242.log.gz 2024-03-16T20:46:43.471 INFO:teuthology.orchestra.run.smithi122.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34681.log.gz 2024-03-16T20:46:43.471 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35606.log 2024-03-16T20:46:43.471 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35943.log 2024-03-16T20:46:43.471 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35606.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36712.log.gz 2024-03-16T20:46:43.471 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34346.log 2024-03-16T20:46:43.471 INFO:teuthology.orchestra.run.smithi122.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35606.log.gz 2024-03-16T20:46:43.471 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.35943.log: -- /var/log/ceph/ceph-client.admin.35338.log 2024-03-16T20:46:43.477 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35788.log 2024-03-16T20:46:43.477 INFO:teuthology.orchestra.run.smithi122.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35943.log.gz 2024-03-16T20:46:43.477 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35338.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34346.log.gz 2024-03-16T20:46:43.477 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36294.log 2024-03-16T20:46:43.477 INFO:teuthology.orchestra.run.smithi122.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35338.log.gz 2024-03-16T20:46:43.477 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32774.log 2024-03-16T20:46:43.477 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36294.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35788.log.gz 2024-03-16T20:46:43.478 INFO:teuthology.orchestra.run.smithi122.stderr: 92.5%gzip -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-03-16T20:46:43.478 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33174.log 2024-03-16T20:46:43.478 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32774.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.36294.log.gz 2024-03-16T20:46:43.478 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-03-16T20:46:43.478 INFO:teuthology.orchestra.run.smithi122.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32774.log.gz 2024-03-16T20:46:43.478 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36376.log 2024-03-16T20:46:43.478 INFO:teuthology.orchestra.run.smithi122.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33174.log.gz 2024-03-16T20:46:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34785.log 2024-03-16T20:46:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32959.log 2024-03-16T20:46:43.479 INFO:teuthology.orchestra.run.smithi122.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36376.log.gz 2024-03-16T20:46:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36125.log 2024-03-16T20:46:43.479 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34785.log.gz 2024-03-16T20:46:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32907.log 2024-03-16T20:46:43.479 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36151.log 2024-03-16T20:46:43.479 INFO:teuthology.orchestra.run.smithi122.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32959.log.gz 2024-03-16T20:46:43.479 INFO:teuthology.orchestra.run.smithi122.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36125.log.gz 2024-03-16T20:46:43.480 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33358.log 2024-03-16T20:46:43.480 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32907.log.gz 2024-03-16T20:46:43.480 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36203.log 2024-03-16T20:46:43.480 INFO:teuthology.orchestra.run.smithi122.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36151.log.gz 2024-03-16T20:46:43.480 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34372.log 2024-03-16T20:46:43.491 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34941.log 2024-03-16T20:46:43.491 INFO:teuthology.orchestra.run.smithi122.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33358.log.gz 2024-03-16T20:46:43.492 INFO:teuthology.orchestra.run.smithi122.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36203.log.gz 2024-03-16T20:46:43.492 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34450.log 2024-03-16T20:46:43.492 INFO:teuthology.orchestra.run.smithi122.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34372.log.gz 2024-03-16T20:46:43.492 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33461.log 2024-03-16T20:46:43.492 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34138.log 2024-03-16T20:46:43.492 INFO:teuthology.orchestra.run.smithi122.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34941.log.gz 2024-03-16T20:46:43.492 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33461.log: 91.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.34450.log.gz -5 2024-03-16T20:46:43.492 INFO:teuthology.orchestra.run.smithi122.stderr: --verbose -- /var/log/ceph/ceph-client.admin.35443.log 2024-03-16T20:46:43.493 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33461.log.gz 2024-03-16T20:46:43.493 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33200.log 2024-03-16T20:46:43.493 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33877.log 2024-03-16T20:46:43.493 INFO:teuthology.orchestra.run.smithi122.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34138.log.gz 2024-03-16T20:46:43.493 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33200.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35443.log.gz 2024-03-16T20:46:43.493 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37980.log 2024-03-16T20:46:43.493 INFO:teuthology.orchestra.run.smithi122.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33200.log.gz 2024-03-16T20:46:43.493 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37434.log 2024-03-16T20:46:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36099.log 2024-03-16T20:46:43.494 INFO:teuthology.orchestra.run.smithi122.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33877.log.gz 2024-03-16T20:46:43.494 INFO:teuthology.orchestra.run.smithi122.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37980.log.gz 2024-03-16T20:46:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37434.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-03-16T20:46:43.494 INFO:teuthology.orchestra.run.smithi122.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37434.log.gz 2024-03-16T20:46:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35840.log 2024-03-16T20:46:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37460.log 2024-03-16T20:46:43.494 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.35840.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36099.log.gz 2024-03-16T20:46:43.495 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37694.log 2024-03-16T20:46:43.495 INFO:teuthology.orchestra.run.smithi122.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35840.log.gz 2024-03-16T20:46:43.495 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33254.log 2024-03-16T20:46:43.495 INFO:teuthology.orchestra.run.smithi122.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37460.log.gz 2024-03-16T20:46:43.495 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37538.log 2024-03-16T20:46:43.495 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37694.log: /var/log/ceph/ceph-client.admin.33254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35227.log 2024-03-16T20:46:43.495 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37538.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33254.log.gz 2024-03-16T20:46:43.495 INFO:teuthology.orchestra.run.smithi122.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37694.log.gz 2024-03-16T20:46:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34837.log 2024-03-16T20:46:43.496 INFO:teuthology.orchestra.run.smithi122.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37538.log.gz 2024-03-16T20:46:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34889.log 2024-03-16T20:46:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35227.log: /var/log/ceph/ceph-client.admin.34837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37590.log 2024-03-16T20:46:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34889.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34837.log.gz 2024-03-16T20:46:43.496 INFO:teuthology.orchestra.run.smithi122.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35227.log.gz 2024-03-16T20:46:43.496 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-03-16T20:46:43.496 INFO:teuthology.orchestra.run.smithi122.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34889.log.gz 2024-03-16T20:46:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33332.log 2024-03-16T20:46:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37590.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33280.log 2024-03-16T20:46:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33332.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37590.log.gz 2024-03-16T20:46:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35580.log 2024-03-16T20:46:43.497 INFO:teuthology.orchestra.run.smithi122.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33332.log.gz 2024-03-16T20:46:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37245.log 2024-03-16T20:46:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35045.log 2024-03-16T20:46:43.497 INFO:teuthology.orchestra.run.smithi122.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33280.log.gz 2024-03-16T20:46:43.497 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37245.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35580.log.gz 2024-03-16T20:46:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35526.log 2024-03-16T20:46:43.498 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37245.log.gz 2024-03-16T20:46:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37167.log 2024-03-16T20:46:43.498 INFO:teuthology.orchestra.run.smithi122.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35045.log.gz 2024-03-16T20:46:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38030.log 2024-03-16T20:46:43.498 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35526.log: /var/log/ceph/ceph-client.admin.37167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36428.log 2024-03-16T20:46:43.498 INFO:teuthology.orchestra.run.smithi122.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35526.log.gz 2024-03-16T20:46:43.499 INFO:teuthology.orchestra.run.smithi122.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37167.log.gz 2024-03-16T20:46:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36902.log 2024-03-16T20:46:43.499 INFO:teuthology.orchestra.run.smithi122.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38030.log.gz 2024-03-16T20:46:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37394.log 2024-03-16T20:46:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36428.log: /var/log/ceph/ceph-client.admin.36902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37798.log 2024-03-16T20:46:43.499 INFO:teuthology.orchestra.run.smithi122.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36428.log.gz 2024-03-16T20:46:43.499 INFO:teuthology.orchestra.run.smithi122.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.36902.log.gz 2024-03-16T20:46:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33148.log 2024-03-16T20:46:43.499 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36532.log 2024-03-16T20:46:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37798.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37394.log.gz 2024-03-16T20:46:43.500 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33880.log 2024-03-16T20:46:43.500 INFO:teuthology.orchestra.run.smithi122.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37798.log.gz 2024-03-16T20:46:43.515 INFO:teuthology.orchestra.run.smithi122.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33148.log.gz 2024-03-16T20:46:43.515 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36320.log 2024-03-16T20:46:43.515 INFO:teuthology.orchestra.run.smithi122.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36532.log.gz 2024-03-16T20:46:43.515 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33014.log 2024-03-16T20:46:43.515 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33672.log 2024-03-16T20:46:43.515 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33880.log.gz 2024-03-16T20:46:43.516 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33014.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36320.log.gz 2024-03-16T20:46:43.516 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36454.log 2024-03-16T20:46:43.516 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33014.log.gz 2024-03-16T20:46:43.516 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36954.log 2024-03-16T20:46:43.516 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36454.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33672.log.gz 2024-03-16T20:46:43.516 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36820.log 2024-03-16T20:46:43.516 INFO:teuthology.orchestra.run.smithi122.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36454.log.gz 2024-03-16T20:46:43.516 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33459.log 2024-03-16T20:46:43.517 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36954.log: /var/log/ceph/ceph-client.admin.36820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37824.log 2024-03-16T20:46:43.517 INFO:teuthology.orchestra.run.smithi122.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36954.log.gz 2024-03-16T20:46:43.517 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33459.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36820.log.gz 2024-03-16T20:46:43.517 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33384.log 2024-03-16T20:46:43.517 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37824.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33459.log.gz 2024-03-16T20:46:43.517 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37928.log 2024-03-16T20:46:43.517 INFO:teuthology.orchestra.run.smithi122.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37824.log.gz 2024-03-16T20:46:43.517 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37720.log 2024-03-16T20:46:43.518 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36584.log 2024-03-16T20:46:43.518 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33384.log.gz 2024-03-16T20:46:43.518 INFO:teuthology.orchestra.run.smithi122.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37928.log.gz 2024-03-16T20:46:43.518 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34811.log 2024-03-16T20:46:43.518 INFO:teuthology.orchestra.run.smithi122.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37720.log.gz 2024-03-16T20:46:43.518 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35632.log 2024-03-16T20:46:43.518 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34811.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36584.log.gz 2024-03-16T20:46:43.519 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33876.log 2024-03-16T20:46:43.519 INFO:teuthology.orchestra.run.smithi122.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34811.log.gz 2024-03-16T20:46:43.519 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27504.log 2024-03-16T20:46:43.519 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35632.log: /var/log/ceph/ceph-client.admin.33876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37342.log 2024-03-16T20:46:43.519 INFO:teuthology.orchestra.run.smithi122.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35632.log.gz 2024-03-16T20:46:43.519 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33876.log.gz 2024-03-16T20:46:43.519 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.27504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34112.log 2024-03-16T20:46:43.520 INFO:teuthology.orchestra.run.smithi122.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27504.log.gz 2024-03-16T20:46:43.520 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35019.log 2024-03-16T20:46:43.520 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37342.log.gz 2024-03-16T20:46:43.520 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34112.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-03-16T20:46:43.520 INFO:teuthology.orchestra.run.smithi122.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34112.log.gz 2024-03-16T20:46:43.520 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34967.log 2024-03-16T20:46:43.520 INFO:teuthology.orchestra.run.smithi122.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35019.log.gz 2024-03-16T20:46:43.520 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35123.log 2024-03-16T20:46:43.521 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34733.log 2024-03-16T20:46:43.521 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35123.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34967.log.gz 2024-03-16T20:46:43.521 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35071.log 2024-03-16T20:46:43.521 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34733.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35123.log.gz 2024-03-16T20:46:43.521 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35149.log 2024-03-16T20:46:43.521 INFO:teuthology.orchestra.run.smithi122.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34733.log.gz 2024-03-16T20:46:43.521 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33671.log 2024-03-16T20:46:43.522 INFO:teuthology.orchestra.run.smithi122.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35071.log.gz 2024-03-16T20:46:43.522 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35814.log 2024-03-16T20:46:43.522 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35149.log.gz 2024-03-16T20:46:43.522 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34707.log 2024-03-16T20:46:43.522 INFO:teuthology.orchestra.run.smithi122.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33671.log.gz 2024-03-16T20:46:43.522 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33462.log 2024-03-16T20:46:43.522 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34707.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35814.log.gz 2024-03-16T20:46:43.522 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32985.log 2024-03-16T20:46:43.523 INFO:teuthology.orchestra.run.smithi122.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34707.log.gz 2024-03-16T20:46:43.523 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37063.log 2024-03-16T20:46:43.523 INFO:teuthology.orchestra.run.smithi122.stderr: 88.3%/var/log/ceph/ceph-client.admin.32985.log: -- replaced with /var/log/ceph/ceph-client.admin.33462.log.gz 2024-03-16T20:46:43.523 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35391.log 2024-03-16T20:46:43.523 INFO:teuthology.orchestra.run.smithi122.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32985.log.gz 2024-03-16T20:46:43.523 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35364.log 2024-03-16T20:46:43.523 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35391.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37063.log.gz 2024-03-16T20:46:43.523 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-03-16T20:46:43.523 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35364.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35391.log.gz 2024-03-16T20:46:43.524 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33658.log 2024-03-16T20:46:43.524 INFO:teuthology.orchestra.run.smithi122.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35364.log.gz 2024-03-16T20:46:43.524 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36984.log 2024-03-16T20:46:43.524 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35175.log 2024-03-16T20:46:43.524 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9%/var/log/ceph/ceph-client.admin.36984.log: -- replaced with /var/log/ceph/ceph-client.admin.33658.log.gz 2024-03-16T20:46:43.524 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35097.log 2024-03-16T20:46:43.524 INFO:teuthology.orchestra.run.smithi122.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36984.log.gz 2024-03-16T20:46:43.524 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36850.log 2024-03-16T20:46:43.525 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35097.log: /var/log/ceph/ceph-client.admin.35175.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35097.log.gz 2024-03-16T20:46:43.525 INFO:teuthology.orchestra.run.smithi122.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35175.log.gz 2024-03-16T20:46:43.525 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36231.log 2024-03-16T20:46:43.525 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36177.log 2024-03-16T20:46:43.525 INFO:teuthology.orchestra.run.smithi122.stderr: 88.8%/var/log/ceph/ceph-client.admin.36231.log: -- replaced with /var/log/ceph/ceph-client.admin.36850.log.gzgzip 2024-03-16T20:46:43.525 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37512.log 2024-03-16T20:46:43.525 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36231.log.gz 2024-03-16T20:46:43.526 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36177.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36177.log.gz 2024-03-16T20:46:43.526 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34863.log 2024-03-16T20:46:43.526 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34268.log 2024-03-16T20:46:43.526 INFO:teuthology.orchestra.run.smithi122.stderr: 88.9%/var/log/ceph/ceph-client.admin.34863.log: -- replaced with /var/log/ceph/ceph-client.admin.37512.log.gzgzip 2024-03-16T20:46:43.526 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37616.log 2024-03-16T20:46:43.526 INFO:teuthology.orchestra.run.smithi122.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34863.log.gz 2024-03-16T20:46:43.526 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34268.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34268.log.gz 2024-03-16T20:46:43.526 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33306.log 2024-03-16T20:46:43.527 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35286.log 2024-03-16T20:46:43.527 INFO:teuthology.orchestra.run.smithi122.stderr: 88.8%/var/log/ceph/ceph-client.admin.33306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36928.log 2024-03-16T20:46:43.527 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37616.log.gz 2024-03-16T20:46:43.527 INFO:teuthology.orchestra.run.smithi122.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33306.log.gz 2024-03-16T20:46:43.527 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35286.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35286.log.gz 2024-03-16T20:46:43.536 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27522.log 2024-03-16T20:46:43.536 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-03-16T20:46:43.536 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.27522.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-03-16T20:46:43.536 INFO:teuthology.orchestra.run.smithi195.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27522.log.gz 2024-03-16T20:46:43.537 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27673.log 2024-03-16T20:46:43.537 INFO:teuthology.orchestra.run.smithi195.stderr:gzip/var/log/ceph/ceph-osd.6.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.27498.log 2024-03-16T20:46:43.537 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.27673.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-03-16T20:46:43.537 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.27498.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.27673.log.gz 2024-03-16T20:46:43.537 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27642.log 2024-03-16T20:46:43.537 INFO:teuthology.orchestra.run.smithi195.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.27498.log.gz 2024-03-16T20:46:43.537 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27618.log 2024-03-16T20:46:43.537 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.27642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27570.log 2024-03-16T20:46:43.538 INFO:teuthology.orchestra.run.smithi195.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27642.log.gz 2024-03-16T20:46:43.538 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27546.log 2024-03-16T20:46:43.538 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.27618.log: /var/log/ceph/ceph-client.admin.27570.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-03-16T20:46:43.538 INFO:teuthology.orchestra.run.smithi195.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27570.log.gz 2024-03-16T20:46:43.538 INFO:teuthology.orchestra.run.smithi195.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27618.log.gz 2024-03-16T20:46:43.538 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.27546.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-16T20:46:43.538 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-mon.c.log 2024-03-16T20:46:43.538 INFO:teuthology.orchestra.run.smithi195.stderr: -- replaced with /var/log/ceph/ceph-client.admin.27546.log.gz 2024-03-16T20:46:43.539 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-16T20:46:43.539 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-03-16T20:46:43.539 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph.audit.log: 91.3% -- replaced with /var/log/ceph/ceph.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-03-16T20:46:43.539 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-16T20:46:43.539 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-03-16T20:46:43.539 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-mon.b.log: 93.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-16T20:46:43.541 INFO:teuthology.orchestra.run.smithi195.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27594.log 2024-03-16T20:46:43.556 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-osd.4.log: 90.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-03-16T20:46:43.556 INFO:teuthology.orchestra.run.smithi195.stderr:/var/log/ceph/ceph-client.admin.27594.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.27594.log.gz 2024-03-16T20:46:43.560 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35469.log 2024-03-16T20:46:43.560 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37564.log 2024-03-16T20:46:43.560 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35469.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33852.log 2024-03-16T20:46:43.560 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36928.log.gz 2024-03-16T20:46:43.560 INFO:teuthology.orchestra.run.smithi122.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35469.log.gz 2024-03-16T20:46:43.560 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37564.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37564.log.gz 2024-03-16T20:46:43.561 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34190.log 2024-03-16T20:46:43.561 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33869.log 2024-03-16T20:46:43.561 INFO:teuthology.orchestra.run.smithi122.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33852.log.gz 2024-03-16T20:46:43.561 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.34190.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33878.log 2024-03-16T20:46:43.561 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33869.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34190.log.gz 2024-03-16T20:46:43.561 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33869.log.gz 2024-03-16T20:46:43.561 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36610.log 2024-03-16T20:46:43.561 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36073.log 2024-03-16T20:46:43.562 INFO:teuthology.orchestra.run.smithi122.stderr: 88.9%/var/log/ceph/ceph-client.admin.36610.log: -- replaced with /var/log/ceph/ceph-client.admin.33878.log.gz 2024-03-16T20:46:43.562 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35897.log 2024-03-16T20:46:43.562 INFO:teuthology.orchestra.run.smithi122.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36610.log.gz 2024-03-16T20:46:43.562 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36073.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36073.log.gz 2024-03-16T20:46:43.562 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33666.log 2024-03-16T20:46:43.562 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37902.log 2024-03-16T20:46:43.562 INFO:teuthology.orchestra.run.smithi122.stderr: 88.3%/var/log/ceph/ceph-client.admin.33666.log: -- replaced with /var/log/ceph/ceph-client.admin.35897.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34294.log 2024-03-16T20:46:43.562 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-03-16T20:46:43.562 INFO:teuthology.orchestra.run.smithi122.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33666.log.gz 2024-03-16T20:46:43.563 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37902.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37902.log.gz 2024-03-16T20:46:43.563 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-03-16T20:46:43.563 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33874.log 2024-03-16T20:46:43.563 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37279.log 2024-03-16T20:46:43.563 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34294.log.gz 2024-03-16T20:46:43.563 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33874.log: 84.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-03-16T20:46:43.563 INFO:teuthology.orchestra.run.smithi122.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33874.log.gz 2024-03-16T20:46:43.563 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37316.log 2024-03-16T20:46:43.564 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37089.log 2024-03-16T20:46:43.564 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36480.log 2024-03-16T20:46:43.564 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37316.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37279.log.gz 2024-03-16T20:46:43.564 INFO:teuthology.orchestra.run.smithi122.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37316.log.gz 2024-03-16T20:46:43.564 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37089.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37089.log.gz 2024-03-16T20:46:43.564 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33661.log 2024-03-16T20:46:43.564 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33879.log 2024-03-16T20:46:43.564 INFO:teuthology.orchestra.run.smithi122.stderr: 89.4%/var/log/ceph/ceph-client.admin.33661.log: -- replaced with /var/log/ceph/ceph-client.admin.36480.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35260.log 2024-03-16T20:46:43.564 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-03-16T20:46:43.565 INFO:teuthology.orchestra.run.smithi122.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33661.log.gz 2024-03-16T20:46:43.565 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33879.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33879.log.gz 2024-03-16T20:46:43.565 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34424.log 2024-03-16T20:46:43.565 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36402.log 2024-03-16T20:46:43.565 INFO:teuthology.orchestra.run.smithi122.stderr: 88.9%/var/log/ceph/ceph-client.admin.34424.log: -- replaced with /var/log/ceph/ceph-client.admin.35260.log.gz 2024-03-16T20:46:43.565 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36021.log 2024-03-16T20:46:43.565 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34759.log 2024-03-16T20:46:43.565 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36402.log: 89.2%/var/log/ceph/ceph-client.admin.36021.log: -- replaced with /var/log/ceph/ceph-client.admin.34424.log.gz 2024-03-16T20:46:43.566 INFO:teuthology.orchestra.run.smithi122.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36402.log.gz 2024-03-16T20:46:43.566 INFO:teuthology.orchestra.run.smithi122.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36021.log.gz 2024-03-16T20:46:43.566 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34915.log 2024-03-16T20:46:43.566 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33122.log 2024-03-16T20:46:43.566 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34915.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34759.log.gz 2024-03-16T20:46:43.566 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37486.log 2024-03-16T20:46:43.566 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37954.log 2024-03-16T20:46:43.566 INFO:teuthology.orchestra.run.smithi122.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34915.log.gz/var/log/ceph/ceph-client.admin.33122.log: 2024-03-16T20:46:43.567 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37486.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33122.log.gz 2024-03-16T20:46:43.567 INFO:teuthology.orchestra.run.smithi122.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37486.log.gz 2024-03-16T20:46:43.567 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34320.log 2024-03-16T20:46:43.567 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37642.log 2024-03-16T20:46:43.567 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34320.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37954.log.gz 2024-03-16T20:46:43.567 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33458.log 2024-03-16T20:46:43.567 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32933.log 2024-03-16T20:46:43.567 INFO:teuthology.orchestra.run.smithi122.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34320.log.gz 2024-03-16T20:46:43.567 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37642.log: /var/log/ceph/ceph-client.admin.33458.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37642.log.gz 2024-03-16T20:46:43.568 INFO:teuthology.orchestra.run.smithi122.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33458.log.gz 2024-03-16T20:46:43.568 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32830.log 2024-03-16T20:46:43.568 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37746.log 2024-03-16T20:46:43.568 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32830.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32933.log.gz 2024-03-16T20:46:43.568 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35312.log 2024-03-16T20:46:43.568 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37141.log 2024-03-16T20:46:43.568 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37746.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32830.log.gz/var/log/ceph/ceph-client.admin.35312.log: 2024-03-16T20:46:43.568 INFO:teuthology.orchestra.run.smithi122.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37746.log.gz 2024-03-16T20:46:43.568 INFO:teuthology.orchestra.run.smithi122.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35312.log.gz 2024-03-16T20:46:43.569 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37219.log 2024-03-16T20:46:43.569 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33668.log 2024-03-16T20:46:43.569 INFO:teuthology.orchestra.run.smithi122.stderr: 88.2%/var/log/ceph/ceph-client.admin.37219.log: -- replaced with /var/log/ceph/ceph-client.admin.37141.log.gz 2024-03-16T20:46:43.569 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-03-16T20:46:43.569 INFO:teuthology.orchestra.run.smithi122.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37219.log.gz 2024-03-16T20:46:43.569 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33668.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33668.log.gz 2024-03-16T20:46:43.569 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35762.log 2024-03-16T20:46:43.569 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34505.log 2024-03-16T20:46:43.569 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.35762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36346.log 2024-03-16T20:46:43.570 INFO:teuthology.orchestra.run.smithi122.stderr: 88.5%/var/log/ceph/ceph-client.admin.34505.log: -- replaced with /var/log/ceph/ceph-client.admin.35762.log.gz 2024-03-16T20:46:43.570 INFO:teuthology.orchestra.run.smithi122.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34505.log.gz 2024-03-16T20:46:43.570 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37193.log 2024-03-16T20:46:43.570 INFO:teuthology.orchestra.run.smithi122.stderr: 88.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-03-16T20:46:43.570 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33445.log 2024-03-16T20:46:43.570 INFO:teuthology.orchestra.run.smithi122.stderr: 89.2%/var/log/ceph/ceph-client.admin.37193.log: -- replaced with /var/log/ceph/ceph-client.admin.36346.log.gz 2024-03-16T20:46:43.570 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-03-16T20:46:43.570 INFO:teuthology.orchestra.run.smithi122.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37193.log.gz 2024-03-16T20:46:43.570 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33445.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33445.log.gz 2024-03-16T20:46:43.571 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35417.log 2024-03-16T20:46:43.571 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33094.log 2024-03-16T20:46:43.571 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.35417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33226.log 2024-03-16T20:46:43.571 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33094.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35417.log.gz 2024-03-16T20:46:43.571 INFO:teuthology.orchestra.run.smithi122.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33094.log.gz 2024-03-16T20:46:43.571 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37668.log 2024-03-16T20:46:43.571 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35995.log 2024-03-16T20:46:43.571 INFO:teuthology.orchestra.run.smithi122.stderr: 89.6%/var/log/ceph/ceph-client.admin.37668.log: -- replaced with /var/log/ceph/ceph-client.admin.33226.log.gz 2024-03-16T20:46:43.572 INFO:teuthology.orchestra.run.smithi122.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37668.log.gz 2024-03-16T20:46:43.572 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34164.log 2024-03-16T20:46:43.572 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35684.log 2024-03-16T20:46:43.572 INFO:teuthology.orchestra.run.smithi122.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35995.log.gz 2024-03-16T20:46:43.572 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34164.log: 89.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38004.log 2024-03-16T20:46:43.572 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34164.log.gz 2024-03-16T20:46:43.572 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33669.log 2024-03-16T20:46:43.572 INFO:teuthology.orchestra.run.smithi122.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35684.log.gz 2024-03-16T20:46:43.572 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33040.log 2024-03-16T20:46:43.573 INFO:teuthology.orchestra.run.smithi122.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38004.log.gz 2024-03-16T20:46:43.573 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33066.log 2024-03-16T20:46:43.573 INFO:teuthology.orchestra.run.smithi122.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33669.log.gz 2024-03-16T20:46:43.573 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34060.log 2024-03-16T20:46:43.573 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33066.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33040.log.gz 2024-03-16T20:46:43.573 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35969.log 2024-03-16T20:46:43.573 INFO:teuthology.orchestra.run.smithi122.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33066.log.gz 2024-03-16T20:46:43.573 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37772.log 2024-03-16T20:46:43.574 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35969.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34060.log.gz 2024-03-16T20:46:43.574 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.34631.log 2024-03-16T20:46:43.574 INFO:teuthology.orchestra.run.smithi122.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35969.log.gz 2024-03-16T20:46:43.574 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33455.log 2024-03-16T20:46:43.574 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.34631.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.37772.log.gz 2024-03-16T20:46:43.574 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36636.log 2024-03-16T20:46:43.574 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33455.log: 89.6% -- replaced with /var/log/ceph/ceph-client.0.34631.log.gz 2024-03-16T20:46:43.574 INFO:teuthology.orchestra.run.smithi122.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33455.log.gz 2024-03-16T20:46:43.574 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37850.log 2024-03-16T20:46:43.575 INFO:teuthology.orchestra.run.smithi195.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-03-16T20:46:43.588 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27387.log 2024-03-16T20:46:43.588 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36636.log.gz/var/log/ceph/ceph-client.admin.37850.log: 2024-03-16T20:46:43.588 INFO:teuthology.orchestra.run.smithi122.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37115.log 2024-03-16T20:46:43.588 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37850.log.gz 2024-03-16T20:46:43.588 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.27387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35495.log 2024-03-16T20:46:43.588 INFO:teuthology.orchestra.run.smithi122.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.27387.log.gz 2024-03-16T20:46:43.588 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37115.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36767.log 2024-03-16T20:46:43.589 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37115.log.gz 2024-03-16T20:46:43.589 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35552.log 2024-03-16T20:46:43.589 INFO:teuthology.orchestra.run.smithi122.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35495.log.gz 2024-03-16T20:46:43.589 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.0.36767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37876.log 2024-03-16T20:46:43.589 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35552.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35552.log.gz 2024-03-16T20:46:43.589 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34398.log 2024-03-16T20:46:43.589 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37876.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.37876.log.gz 2024-03-16T20:46:43.589 INFO:teuthology.orchestra.run.smithi122.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.0.36767.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35868.log 2024-03-16T20:46:43.589 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-03-16T20:46:43.590 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37011.log 2024-03-16T20:46:43.590 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35868.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34398.log.gz 2024-03-16T20:46:43.590 INFO:teuthology.orchestra.run.smithi122.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35868.log.gz 2024-03-16T20:46:43.590 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27625.log 2024-03-16T20:46:43.590 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37368.log 2024-03-16T20:46:43.590 INFO:teuthology.orchestra.run.smithi122.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-03-16T20:46:43.590 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37011.log: /var/log/ceph/ceph-client.admin.27625.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.27625.log.gz 2024-03-16T20:46:43.590 INFO:teuthology.orchestra.run.smithi122.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37011.log.gz 2024-03-16T20:46:43.591 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36047.log 2024-03-16T20:46:43.591 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34086.log 2024-03-16T20:46:43.591 INFO:teuthology.orchestra.run.smithi122.stderr: 87.9%/var/log/ceph/ceph-client.admin.36047.log: -- replaced with /var/log/ceph/ceph-client.admin.37368.log.gz 2024-03-16T20:46:43.591 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35658.log 2024-03-16T20:46:43.591 INFO:teuthology.orchestra.run.smithi122.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33463.log 2024-03-16T20:46:43.591 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36047.log.gz 2024-03-16T20:46:43.591 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34086.log: /var/log/ceph/ceph-client.admin.35658.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34086.log.gz 2024-03-16T20:46:43.591 INFO:teuthology.orchestra.run.smithi122.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35658.log.gz 2024-03-16T20:46:43.592 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36506.log 2024-03-16T20:46:43.592 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35736.log 2024-03-16T20:46:43.592 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36506.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33463.log.gz 2024-03-16T20:46:43.592 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37037.log 2024-03-16T20:46:43.592 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35710.log 2024-03-16T20:46:43.592 INFO:teuthology.orchestra.run.smithi122.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36506.log.gz 2024-03-16T20:46:43.592 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35736.log: /var/log/ceph/ceph-client.admin.37037.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22579.log 2024-03-16T20:46:43.592 INFO:teuthology.orchestra.run.smithi122.stderr: 88.9% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35736.log.gz 2024-03-16T20:46:43.592 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37037.log.gz 2024-03-16T20:46:43.593 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36876.log 2024-03-16T20:46:43.593 INFO:teuthology.orchestra.run.smithi122.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35710.log.gz 2024-03-16T20:46:43.593 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34993.log 2024-03-16T20:46:43.593 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph.tmp-client.admin.22579.log: /var/log/ceph/ceph-client.admin.36876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36662.log 2024-03-16T20:46:43.593 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.22579.log.gz 2024-03-16T20:46:43.593 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34993.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36259.log 2024-03-16T20:46:43.593 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36876.log.gz 2024-03-16T20:46:43.593 INFO:teuthology.orchestra.run.smithi122.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34993.log.gz 2024-03-16T20:46:43.593 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36662.log: 93.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-03-16T20:46:43.594 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33464.log 2024-03-16T20:46:43.594 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34216.log 2024-03-16T20:46:43.594 INFO:teuthology.orchestra.run.smithi122.stderr: 91.4%/var/log/ceph/ceph-client.admin.33464.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36259.log.gz 2024-03-16T20:46:43.594 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36558.log 2024-03-16T20:46:43.594 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36662.log.gz 2024-03-16T20:46:43.594 INFO:teuthology.orchestra.run.smithi122.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33464.log.gz 2024-03-16T20:46:43.594 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.34216.log: /var/log/ceph/ceph-client.admin.33410.log 2024-03-16T20:46:43.594 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36558.log: gzip 88.4% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.34216.log.gz -- 2024-03-16T20:46:43.595 INFO:teuthology.orchestra.run.smithi122.stderr: /var/log/ceph/ceph-client.admin.32861.log 2024-03-16T20:46:43.595 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33410.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36558.log.gz 2024-03-16T20:46:43.595 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35201.log 2024-03-16T20:46:43.595 INFO:teuthology.orchestra.run.smithi122.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33410.log.gz 2024-03-16T20:46:43.595 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33670.log 2024-03-16T20:46:43.595 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35201.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32861.log.gz 2024-03-16T20:46:43.595 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33670.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35201.log.gz 2024-03-16T20:46:43.595 INFO:teuthology.orchestra.run.smithi122.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33670.log.gz 2024-03-16T20:46:43.611 INFO:teuthology.orchestra.run.smithi122.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-03-16T20:46:43.613 INFO:teuthology.orchestra.run.smithi195.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-03-16T20:46:43.613 INFO:teuthology.orchestra.run.smithi122.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-03-16T20:46:43.622 INFO:teuthology.orchestra.run.smithi195.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-03-16T20:46:43.626 INFO:teuthology.orchestra.run.smithi195.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-03-16T20:46:43.633 INFO:teuthology.orchestra.run.smithi195.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-03-16T20:46:43.637 INFO:teuthology.orchestra.run.smithi122.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-03-16T20:46:43.657 INFO:teuthology.orchestra.run.smithi122.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-03-16T20:46:43.658 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-03-16T20:46:43.659 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m0.208s 2024-03-16T20:46:43.659 INFO:teuthology.orchestra.run.smithi122.stderr:user 0m0.837s 2024-03-16T20:46:43.659 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m0.107s 2024-03-16T20:46:43.660 INFO:teuthology.orchestra.run.smithi195.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-03-16T20:46:43.684 INFO:teuthology.orchestra.run.smithi195.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-03-16T20:46:43.686 INFO:teuthology.orchestra.run.smithi195.stderr: 2024-03-16T20:46:43.686 INFO:teuthology.orchestra.run.smithi195.stderr:real 0m0.182s 2024-03-16T20:46:43.686 INFO:teuthology.orchestra.run.smithi195.stderr:user 0m0.686s 2024-03-16T20:46:43.686 INFO:teuthology.orchestra.run.smithi195.stderr:sys 0m0.060s 2024-03-16T20:46:43.686 INFO:tasks.ceph:Archiving logs... 2024-03-16T20:46:43.687 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602163/remote/smithi122/log 2024-03-16T20:46:43.688 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T20:46:43.928 DEBUG:teuthology.misc:Transferring archived files from smithi195:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602163/remote/smithi195/log 2024-03-16T20:46:43.929 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-16T20:46:44.019 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-16T20:46:44.038 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-16T20:46:44.038 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T20:46:44.041 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-16T20:46:44.118 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-16T20:46:44.127 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-16T20:46:44.127 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T20:46:44.129 DEBUG:teuthology.orchestra.run.smithi195:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-16T20:46:44.140 INFO:teuthology.orchestra.run.smithi122.stdout: remote refid st t when poll reach delay offset jitter 2024-03-16T20:46:44.140 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================== 2024-03-16T20:46:44.140 INFO:teuthology.orchestra.run.smithi122.stdout:*hv01.front.sepi 44.4.53.2 2 u 59 64 37 0.083 -0.715 1.386 2024-03-16T20:46:44.140 INFO:teuthology.orchestra.run.smithi122.stdout:+hv02.front.sepi 63.231.80.2 3 u 60 64 37 0.110 -0.486 1.572 2024-03-16T20:46:44.140 INFO:teuthology.orchestra.run.smithi122.stdout:+hv03.front.sepi 74.6.168.72 3 u 62 64 37 0.087 -2.368 1.484 2024-03-16T20:46:44.141 INFO:teuthology.orchestra.run.smithi122.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T20:46:44.142 INFO:teuthology.orchestra.run.smithi195.stdout: remote refid st t when poll reach delay offset jitter 2024-03-16T20:46:44.142 INFO:teuthology.orchestra.run.smithi195.stdout:============================================================================== 2024-03-16T20:46:44.142 INFO:teuthology.orchestra.run.smithi195.stdout:*hv01.front.sepi 44.4.53.2 2 u 55 64 37 0.078 -0.012 1.121 2024-03-16T20:46:44.142 INFO:teuthology.orchestra.run.smithi195.stdout:+hv02.front.sepi 63.231.80.2 3 u 52 64 37 0.110 +0.265 1.329 2024-03-16T20:46:44.142 INFO:teuthology.orchestra.run.smithi195.stdout:+hv03.front.sepi 74.6.168.72 3 u 57 64 37 0.136 -1.686 1.280 2024-03-16T20:46:44.143 INFO:teuthology.orchestra.run.smithi195.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-16T20:46:44.143 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-16T20:46:44.153 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-16T20:46:44.153 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-16T20:46:44.162 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-16T20:46:44.171 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-16T20:46:44.179 INFO:teuthology.task.internal:Duration was 747.297235 seconds 2024-03-16T20:46:44.179 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-16T20:46:44.189 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-16T20:46:44.189 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T20:46:44.191 DEBUG:teuthology.orchestra.run.smithi195:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-16T20:46:44.228 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-16T20:46:44.229 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-03-16T20:46:44.229 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T20:46:44.279 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi195.front.sepia.ceph.com 2024-03-16T20:46:44.279 DEBUG:teuthology.orchestra.run.smithi195:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-16T20:46:44.289 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-16T20:46:44.289 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T20:46:44.325 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-16T20:46:44.358 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-16T20:46:44.358 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T20:46:44.391 DEBUG:teuthology.orchestra.run.smithi195:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-16T20:46:44.459 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-16T20:46:44.468 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-16T20:46:44.469 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T20:46:44.480 DEBUG:teuthology.orchestra.run.smithi195:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-16T20:46:44.490 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-16T20:46:44.500 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T20:46:44.523 DEBUG:teuthology.orchestra.run.smithi195:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-16T20:46:44.531 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-03-16T20:46:44.542 INFO:teuthology.orchestra.run.smithi195.stdout:kernel.core_pattern = core 2024-03-16T20:46:44.566 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T20:46:44.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T20:46:44.589 DEBUG:teuthology.orchestra.run.smithi195:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-16T20:46:44.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-16T20:46:44.613 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-16T20:46:44.623 INFO:teuthology.task.internal:Transferring archived files... 2024-03-16T20:46:44.624 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602163/remote/smithi122 2024-03-16T20:46:44.624 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T20:46:44.652 DEBUG:teuthology.misc:Transferring archived files from smithi195:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602163/remote/smithi195 2024-03-16T20:46:44.652 DEBUG:teuthology.orchestra.run.smithi195:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-16T20:46:44.674 INFO:teuthology.task.internal:Removing archive directory... 2024-03-16T20:46:44.674 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T20:46:44.691 DEBUG:teuthology.orchestra.run.smithi195:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-16T20:46:44.721 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-16T20:46:44.733 INFO:teuthology.task.internal:Not uploading archives. 2024-03-16T20:46:44.733 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-16T20:46:44.743 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-16T20:46:44.744 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T20:46:44.746 DEBUG:teuthology.orchestra.run.smithi195:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-16T20:46:44.749 INFO:teuthology.orchestra.run.smithi122.stdout: 658616 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 16 20:46 /home/ubuntu/cephtest 2024-03-16T20:46:44.769 INFO:teuthology.orchestra.run.smithi195.stdout: 658634 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 16 20:46 /home/ubuntu/cephtest 2024-03-16T20:46:44.771 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-16T20:46:44.781 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-16T20:46:44.801 INFO:teuthology.nuke:Checking targets against current locks 2024-03-16T20:46:44.827 DEBUG:teuthology.nuke:shortname: smithi122 2024-03-16T20:46:44.827 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T20:46:44.853 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602163', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 20:25:52.594617', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEllsH28MeQ1SW8dxOYFtPgzrJX/rVVTPJCd3FSmYFmusoy8payGfi9Hln4nFDY1GMRB7WRpogcPhOEfv/LrjLw='} 2024-03-16T20:46:44.856 DEBUG:teuthology.nuke:shortname: smithi195 2024-03-16T20:46:44.856 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-16T20:46:44.875 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi195.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-03-15_11:44:52-fs-wip-lusov-qdb-squid-distro-default-smithi/7602163', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-16 20:25:52.596358', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGktsF1WOWYs6IgkXWlub27PjDKbkq2fNseBYcLGWD6zOSTVzBaG1oqXQzKXi3cW65Xd617wzRsqSvsyiHVh6yc='} 2024-03-16T20:46:44.904 INFO:teuthology.orchestra.console.smithi122:Power off 2024-03-16T20:46:44.905 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T20:46:44.929 DEBUG:teuthology.orchestra.console.smithi122:power off output: Chassis Power Control: Down/Off 2024-03-16T20:46:44.930 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T20:46:44.945 INFO:teuthology.orchestra.console.smithi195:Power off 2024-03-16T20:46:44.946 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-16T20:46:44.956 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-03-16T20:46:44.970 DEBUG:teuthology.orchestra.console.smithi195:power off output: Chassis Power Control: Down/Off 2024-03-16T20:46:44.970 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T20:46:44.992 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2024-03-16T20:46:45.678 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-03-16T20:46:45.679 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-03-16T20:46:45.679 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-03-16T20:46:48.956 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T20:46:48.992 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T20:46:49.071 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is on 2024-03-16T20:46:49.103 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2024-03-16T20:46:53.075 DEBUG:teuthology.orchestra.console.smithi122:pexpect command: ipmitool -H smithi122.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T20:46:53.104 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T20:46:53.185 DEBUG:teuthology.orchestra.console.smithi122:check power output: Chassis Power is off 2024-03-16T20:46:53.213 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is on 2024-03-16T20:46:53.287 INFO:teuthology.orchestra.console.smithi122:Power off completed 2024-03-16T20:46:57.216 DEBUG:teuthology.orchestra.console.smithi195:pexpect command: ipmitool -H smithi195.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-16T20:46:57.329 DEBUG:teuthology.orchestra.console.smithi195:check power output: Chassis Power is off 2024-03-16T20:46:57.429 INFO:teuthology.orchestra.console.smithi195:Power off completed 2024-03-16T20:46:57.534 INFO:teuthology.run:Summary data: description: fs/shell/{begin/{0-install 1-ceph 2-logrotate} clusters/1-mds-1-client-coloc conf/{client mds mon osd} distro/ubuntu_latest mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/cephfs-shell} duration: 747.2972354888916 failure_reason: 'Test failure: test_cd_with_args (tasks.cephfs.test_cephfs_shell.TestCD)' flavor: default owner: scheduled_leonidus@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=37190baf060548af9205195780300456 status: fail success: false 2024-03-16T20:46:57.534 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-16T20:46:57.616 INFO:teuthology.run:FAIL